2023-12-12T09:35:15.100 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-12T09:35:15.100 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} --owner mchangir --archive /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220 --name mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi -- /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/orig.config.yaml 2023-12-12T09:35:15.159 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-12T09:35:15.251 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220 branch: wip-mchangir-testing-20231205.064351 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} email: mchangir@redhat.com first_in_suite: false job_id: '7488220' kernel: client: sha1: distro flavor: default kdb: true sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-mchangir-testing-20231205.064351 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-mchangir-testing-20231205.064351 sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 owner: mchangir priority: 1000 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5473 sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 sleep_before_teardown: 0 subset: 600/1024 suite: fs suite_branch: wip-mchangir-testing-20231205.064351 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi173.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-12_08:46:52 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-12T09:35:15.251 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa; will attempt to use it 2023-12-12T09:35:15.252 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks 2023-12-12T09:35:15.252 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-12T09:35:15.254 INFO:teuthology.task.internal:Checking packages... 2023-12-12T09:35:15.277 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9' 2023-12-12T09:35:15.278 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-12T09:35:15.278 INFO:teuthology.packaging:ref: None 2023-12-12T09:35:15.278 INFO:teuthology.packaging:tag: None 2023-12-12T09:35:15.278 INFO:teuthology.packaging:branch: wip-mchangir-testing-20231205.064351 2023-12-12T09:35:15.278 INFO:teuthology.packaging:sha1: f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:35:15.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-mchangir-testing-20231205.064351 2023-12-12T09:35:15.515 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-96.gf1fbf3c5 2023-12-12T09:35:15.516 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-12T09:35:15.523 INFO:teuthology.task.internal:no buildpackages task found 2023-12-12T09:35:15.523 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-12T09:35:15.553 INFO:teuthology.task.internal:Saving configuration 2023-12-12T09:35:15.580 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-12T09:35:15.587 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-12T09:35:15.612 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:29:39.006552', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-12T09:35:15.635 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:29:39.005028', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-12T09:35:15.660 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:29:39.007693', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-12T09:35:15.661 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-12T09:35:15.667 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-12-12T09:35:15.667 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-12-12T09:35:15.667 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-12-12T09:35:15.667 INFO:teuthology.run_tasks:Running task console_log... 2023-12-12T09:35:15.773 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8d74456e50>, signals=[15]) 2023-12-12T09:35:15.774 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-12T09:35:15.779 INFO:teuthology.task.internal:Opening connections... 2023-12-12T09:35:15.779 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T09:35:15.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:35:15.853 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2023-12-12T09:35:15.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:35:15.924 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2023-12-12T09:35:15.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:35:15.991 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-12T09:35:16.004 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2023-12-12T09:35:16.026 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2023-12-12T09:35:16.027 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2023-12-12T09:35:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="Red Hat Enterprise Linux" 2023-12-12T09:35:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="8.6 (Ootpa)" 2023-12-12T09:35:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:ID="rhel" 2023-12-12T09:35:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="fedora" 2023-12-12T09:35:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="8.6" 2023-12-12T09:35:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el8" 2023-12-12T09:35:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-12T09:35:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2023-12-12T09:35:16.087 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-12T09:35:16.088 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://www.redhat.com/" 2023-12-12T09:35:16.088 INFO:teuthology.orchestra.run.smithi042.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-12T09:35:16.088 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-12T09:35:16.088 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:35:16.088 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-12T09:35:16.088 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-12T09:35:16.088 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-12T09:35:16.088 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-12T09:35:16.089 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2023-12-12T09:35:16.109 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2023-12-12T09:35:16.130 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2023-12-12T09:35:16.130 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2023-12-12T09:35:16.193 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="Red Hat Enterprise Linux" 2023-12-12T09:35:16.193 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="8.6 (Ootpa)" 2023-12-12T09:35:16.193 INFO:teuthology.orchestra.run.smithi111.stdout:ID="rhel" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="fedora" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="8.6" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el8" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://www.redhat.com/" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-12T09:35:16.194 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-12T09:35:16.195 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2023-12-12T09:35:16.217 DEBUG:teuthology.orchestra.run.smithi173:> uname -m 2023-12-12T09:35:16.238 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2023-12-12T09:35:16.238 DEBUG:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="Red Hat Enterprise Linux" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="8.6 (Ootpa)" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:ID="rhel" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE="fedora" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="8.6" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:PLATFORM_ID="platform:el8" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:ANSI_COLOR="0;31" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://www.redhat.com/" 2023-12-12T09:35:16.298 INFO:teuthology.orchestra.run.smithi173.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-12T09:35:16.299 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-12T09:35:16.299 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:35:16.299 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-12T09:35:16.299 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-12T09:35:16.299 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-12T09:35:16.299 INFO:teuthology.orchestra.run.smithi173.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-12T09:35:16.300 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2023-12-12T09:35:16.325 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-12T09:35:16.336 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-12T09:35:16.343 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-12T09:35:16.343 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2023-12-12T09:35:16.346 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2023-12-12T09:35:16.348 DEBUG:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2023-12-12T09:35:16.367 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-12T09:35:16.373 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-12T09:35:16.373 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2023-12-12T09:35:16.406 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2023-12-12T09:35:16.412 DEBUG:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2023-12-12T09:35:16.443 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-12T09:35:16.516 INFO:teuthology.run_tasks:Running task kernel... 2023-12-12T09:35:16.532 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'} 2023-12-12T09:35:16.532 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-12-12T09:35:16.532 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-12-12T09:35:16.532 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-12T09:35:16.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:35:16.553 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2023-12-12T09:35:16.613 INFO:teuthology.orchestra.run.smithi042.stdout:4.18.0-372.9.1.el8.x86_64 2023-12-12T09:35:16.613 INFO:teuthology.task.kernel:Running kernel on smithi042: 4.18.0-372.9.1.el8.x86_64 2023-12-12T09:35:16.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum install -y kernel 2023-12-12T09:35:17.913 INFO:teuthology.orchestra.run.smithi042.stdout:Updating Subscription Management repositories. 2023-12-12T09:35:17.913 INFO:teuthology.orchestra.run.smithi042.stdout:Unable to read consumer identity 2023-12-12T09:35:19.371 INFO:teuthology.orchestra.run.smithi042.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-12-12T09:35:20.146 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-12-12T09:35:23.822 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 450 kB/s | 24 kB 00:00 2023-12-12T09:35:24.574 INFO:teuthology.orchestra.run.smithi042.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-12-12T09:35:24.615 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2023-12-12T09:35:24.616 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2023-12-12T09:35:24.616 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2023-12-12T09:35:24.733 DEBUG:teuthology.orchestra.run.smithi042:> echo no | sudo yum reinstall kernel || true 2023-12-12T09:35:25.082 INFO:teuthology.orchestra.run.smithi042.stdout:Updating Subscription Management repositories. 2023-12-12T09:35:25.082 INFO:teuthology.orchestra.run.smithi042.stdout:Unable to read consumer identity 2023-12-12T09:35:25.352 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:02 ago on Tue 12 Dec 2023 09:35:23 AM UTC. 2023-12-12T09:35:25.397 INFO:teuthology.orchestra.run.smithi042.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-12T09:35:25.457 INFO:teuthology.orchestra.run.smithi042.stderr:Error: No packages marked for reinstall. 2023-12-12T09:35:25.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum reinstall -y kernel || true 2023-12-12T09:35:25.852 INFO:teuthology.orchestra.run.smithi042.stdout:Updating Subscription Management repositories. 2023-12-12T09:35:25.852 INFO:teuthology.orchestra.run.smithi042.stdout:Unable to read consumer identity 2023-12-12T09:35:26.119 INFO:teuthology.orchestra.run.smithi042.stdout:Last metadata expiration check: 0:00:03 ago on Tue 12 Dec 2023 09:35:23 AM UTC. 2023-12-12T09:35:26.164 INFO:teuthology.orchestra.run.smithi042.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-12T09:35:26.189 INFO:teuthology.orchestra.run.smithi042.stderr:Error: No packages marked for reinstall. 2023-12-12T09:35:26.242 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q kernel | sort -rV | head -n 1 2023-12-12T09:35:26.324 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-12-12T09:35:26.325 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-12-12T09:35:26.325 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-12-12T09:35:26.325 DEBUG:teuthology.parallel:result is None 2023-12-12T09:35:26.325 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-12T09:35:26.333 INFO:teuthology.task.internal:Creating test directory... 2023-12-12T09:35:26.333 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-12T09:35:26.335 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-12T09:35:26.337 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-12T09:35:26.357 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-12T09:35:26.363 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-12T09:35:26.370 INFO:teuthology.task.internal:Creating archive directory... 2023-12-12T09:35:26.370 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-12T09:35:26.393 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-12T09:35:26.398 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-12T09:35:26.439 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-12T09:35:26.446 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-12T09:35:26.446 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-12T09:35:26.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:35:26.492 DEBUG:teuthology.orchestra.run.smithi173:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-12T09:35:26.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:35:26.512 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-12T09:35:26.514 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-12T09:35:26.535 DEBUG:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-12T09:35:26.555 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:35:26.566 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:35:26.586 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:35:26.587 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:35:26.601 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:35:26.615 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-12T09:35:26.617 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-12T09:35:26.624 INFO:teuthology.task.internal:Configuring sudo... 2023-12-12T09:35:26.624 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-12T09:35:26.626 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-12T09:35:26.629 DEBUG:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-12T09:35:26.673 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-12T09:35:26.683 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-12T09:35:26.684 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-12T09:35:26.694 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-12T09:35:26.698 DEBUG:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-12T09:35:26.730 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:35:26.794 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:35:26.858 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:35:26.900 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:35:26.943 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:35:26.944 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-12T09:35:27.012 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:35:27.054 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:35:27.107 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:35:27.192 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:35:27.237 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:35:27.237 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-12T09:35:27.306 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:35:27.353 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-12T09:35:27.416 DEBUG:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:35:27.464 DEBUG:teuthology.orchestra.run.smithi173:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-12T09:35:27.506 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:35:27.506 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-12T09:35:27.575 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2023-12-12T09:35:27.578 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2023-12-12T09:35:27.580 DEBUG:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2023-12-12T09:35:27.645 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T09:35:27.674 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T09:35:27.683 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T09:35:28.009 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-12T09:35:28.018 INFO:teuthology.task.internal:Starting timer... 2023-12-12T09:35:28.018 INFO:teuthology.run_tasks:Running task pcp... 2023-12-12T09:35:28.030 INFO:teuthology.run_tasks:Running task selinux... 2023-12-12T09:35:28.039 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-12-12T09:35:28.040 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2023-12-12T09:35:28.104 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2023-12-12T09:35:28.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2023-12-12T09:35:28.171 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2023-12-12T09:35:28.173 DEBUG:teuthology.orchestra.run.smithi173:> sudo service auditd rotate 2023-12-12T09:35:28.248 INFO:teuthology.orchestra.run.smithi173.stdout:Rotating logs: 2023-12-12T09:35:28.250 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-12T09:35:28.250 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2023-12-12T09:35:28.274 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2023-12-12T09:35:28.275 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2023-12-12T09:35:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2023-12-12T09:35:28.304 DEBUG:teuthology.orchestra.run.smithi173:> /usr/sbin/getenforce 2023-12-12T09:35:28.328 INFO:teuthology.orchestra.run.smithi173.stdout:Permissive 2023-12-12T09:35:28.328 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi173.front.sepia.ceph.com': 'permissive'} 2023-12-12T09:35:28.329 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T09:35:28.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:35:28.359 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T09:35:28.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:35:28.387 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T09:35:28.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T09:35:28.415 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-12T09:35:28.415 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2023-12-12T09:35:28.453 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2023-12-12T09:35:28.481 DEBUG:teuthology.orchestra.run.smithi173:> sudo /usr/sbin/setenforce permissive 2023-12-12T09:35:28.508 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-12T09:35:28.524 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-12T09:35:28.531 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-12T09:35:28.694 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-12T09:35:28.716 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-12T09:35:28.717 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi111.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-12T09:45:36.262 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2023-12-12T09:45:36.263 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2023-12-12T09:45:36.265 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:45:36.335 DEBUG:teuthology.orchestra.run.smithi042:> true 2023-12-12T09:45:36.405 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2023-12-12T09:45:36.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-12T09:45:36.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:45:36.476 DEBUG:teuthology.orchestra.run.smithi111:> true 2023-12-12T09:45:36.545 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-12-12T09:45:36.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-12-12T09:45:36.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-12T09:45:36.615 DEBUG:teuthology.orchestra.run.smithi173:> true 2023-12-12T09:45:36.640 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi173.front.sepia.ceph.com' 2023-12-12T09:45:36.641 INFO:teuthology.run_tasks:Running task clock... 2023-12-12T09:45:36.653 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-12T09:45:36.653 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-12T09:45:36.653 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T09:45:36.656 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-12T09:45:36.656 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T09:45:36.658 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-12T09:45:36.658 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T09:45:36.690 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-12T09:45:36.691 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-12T09:45:36.706 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-12T09:45:36.707 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-12T09:45:36.713 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-12T09:45:36.728 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-12T09:45:36.780 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2023-12-12T09:45:36.793 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2023-12-12T09:45:36.794 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2023-12-12T09:45:36.806 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2023-12-12T09:45:36.807 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: ntpd: command not found 2023-12-12T09:45:36.809 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-12T09:45:36.819 INFO:teuthology.orchestra.run.smithi173.stdout:506 Cannot talk to daemon 2023-12-12T09:45:36.821 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-12T09:45:36.824 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-12T09:45:36.833 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-12T09:45:36.836 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-12T09:45:36.847 INFO:teuthology.orchestra.run.smithi173.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-12T09:45:36.894 INFO:teuthology.orchestra.run.smithi042.stderr:bash: ntpq: command not found 2023-12-12T09:45:36.897 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T09:45:36.897 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2023-12-12T09:45:36.897 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.897 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.897 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.897 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.908 INFO:teuthology.orchestra.run.smithi111.stderr:bash: ntpq: command not found 2023-12-12T09:45:36.911 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T09:45:36.912 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2023-12-12T09:45:36.912 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.912 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.912 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.912 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.918 INFO:teuthology.orchestra.run.smithi173.stderr:bash: ntpq: command not found 2023-12-12T09:45:36.921 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T09:45:36.921 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2023-12-12T09:45:36.922 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.922 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.922 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.922 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T09:45:36.923 INFO:teuthology.run_tasks:Running task pexec... 2023-12-12T09:45:36.932 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-12T09:45:36.933 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T09:45:36.933 DEBUG:teuthology.orchestra.run.smithi042:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-12T09:45:36.933 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi111.front.sepia.ceph.com 2023-12-12T09:45:36.933 DEBUG:teuthology.orchestra.run.smithi111:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-12T09:45:36.934 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi173.front.sepia.ceph.com 2023-12-12T09:45:36.934 DEBUG:teuthology.orchestra.run.smithi173:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-12T09:45:37.323 INFO:teuthology.orchestra.run.smithi111.stdout:Updating Subscription Management repositories. 2023-12-12T09:45:37.330 INFO:teuthology.orchestra.run.smithi042.stdout:Updating Subscription Management repositories. 2023-12-12T09:45:37.335 INFO:teuthology.orchestra.run.smithi173.stdout:Updating Subscription Management repositories. 2023-12-12T09:45:38.040 INFO:teuthology.orchestra.run.smithi042.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 139 kB/s | 4.5 kB 00:00 2023-12-12T09:45:38.048 INFO:teuthology.orchestra.run.smithi111.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 145 kB/s | 4.1 kB 00:00 2023-12-12T09:45:38.051 INFO:teuthology.orchestra.run.smithi173.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 127 kB/s | 4.1 kB 00:00 2023-12-12T09:45:38.249 INFO:teuthology.orchestra.run.smithi111.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 157 kB/s | 4.5 kB 00:00 2023-12-12T09:45:38.263 INFO:teuthology.orchestra.run.smithi173.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 171 kB/s | 4.5 kB 00:00 2023-12-12T09:45:38.372 INFO:teuthology.orchestra.run.smithi042.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 131 kB/s | 4.1 kB 00:00 2023-12-12T09:45:40.165 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2023-12-12T09:45:40.166 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2023-12-12T09:45:40.166 INFO:teuthology.orchestra.run.smithi173.stdout: Package Architecture Version Repository Size 2023-12-12T09:45:40.166 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2023-12-12T09:45:40.166 INFO:teuthology.orchestra.run.smithi173.stdout:Resetting modules: 2023-12-12T09:45:40.166 INFO:teuthology.orchestra.run.smithi173.stdout: container-tools 2023-12-12T09:45:40.166 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:45:40.166 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2023-12-12T09:45:40.166 INFO:teuthology.orchestra.run.smithi173.stdout:================================================================================ 2023-12-12T09:45:40.166 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:45:40.201 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-12-12T09:45:40.202 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2023-12-12T09:45:40.202 INFO:teuthology.orchestra.run.smithi111.stdout: Package Architecture Version Repository Size 2023-12-12T09:45:40.202 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2023-12-12T09:45:40.202 INFO:teuthology.orchestra.run.smithi111.stdout:Resetting modules: 2023-12-12T09:45:40.202 INFO:teuthology.orchestra.run.smithi111.stdout: container-tools 2023-12-12T09:45:40.202 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:45:40.202 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2023-12-12T09:45:40.202 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2023-12-12T09:45:40.202 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:45:40.526 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2023-12-12T09:45:40.527 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2023-12-12T09:45:40.527 INFO:teuthology.orchestra.run.smithi042.stdout: Package Architecture Version Repository Size 2023-12-12T09:45:40.527 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2023-12-12T09:45:40.527 INFO:teuthology.orchestra.run.smithi042.stdout:Resetting modules: 2023-12-12T09:45:40.527 INFO:teuthology.orchestra.run.smithi042.stdout: container-tools 2023-12-12T09:45:40.527 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:45:40.527 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2023-12-12T09:45:40.527 INFO:teuthology.orchestra.run.smithi042.stdout:================================================================================ 2023-12-12T09:45:40.527 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:45:44.396 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-12-12T09:45:44.397 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2023-12-12T09:45:44.465 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2023-12-12T09:45:44.860 INFO:teuthology.orchestra.run.smithi111.stdout:Updating Subscription Management repositories. 2023-12-12T09:45:44.868 INFO:teuthology.orchestra.run.smithi173.stdout:Updating Subscription Management repositories. 2023-12-12T09:45:44.979 INFO:teuthology.orchestra.run.smithi042.stdout:Updating Subscription Management repositories. 2023-12-12T09:45:45.616 INFO:teuthology.orchestra.run.smithi111.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 149 kB/s | 4.1 kB 00:00 2023-12-12T09:45:45.617 INFO:teuthology.orchestra.run.smithi173.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 142 kB/s | 4.1 kB 00:00 2023-12-12T09:45:45.697 INFO:teuthology.orchestra.run.smithi042.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 154 kB/s | 4.5 kB 00:00 2023-12-12T09:45:45.839 INFO:teuthology.orchestra.run.smithi173.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 133 kB/s | 4.5 kB 00:00 2023-12-12T09:45:45.839 INFO:teuthology.orchestra.run.smithi111.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 137 kB/s | 4.5 kB 00:00 2023-12-12T09:45:46.015 INFO:teuthology.orchestra.run.smithi042.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 132 kB/s | 4.1 kB 00:00 2023-12-12T09:45:47.735 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2023-12-12T09:45:47.808 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-12-12T09:45:47.826 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================================================ 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================================================ 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout:Installing group/module packages: 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-12T09:45:47.827 INFO:teuthology.orchestra.run.smithi173.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout:Installing module profiles: 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout: container-tools/common 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout:Enabling module streams: 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout: container-tools rhel8 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================================================ 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout:Install 10 Packages 2023-12-12T09:45:47.828 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:45:47.829 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 18 M 2023-12-12T09:45:47.829 INFO:teuthology.orchestra.run.smithi173.stdout:Installed size: 65 M 2023-12-12T09:45:47.829 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================================================ 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================================================ 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout:Installing group/module packages: 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-12T09:45:47.902 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout:Installing module profiles: 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout: container-tools/common 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout:Enabling module streams: 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout: container-tools rhel8 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================================================ 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout:Install 10 Packages 2023-12-12T09:45:47.903 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:45:47.904 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 18 M 2023-12-12T09:45:47.904 INFO:teuthology.orchestra.run.smithi111.stdout:Installed size: 65 M 2023-12-12T09:45:47.904 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2023-12-12T09:45:47.927 INFO:teuthology.orchestra.run.smithi173.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 956 kB/s | 94 kB 00:00 2023-12-12T09:45:47.954 INFO:teuthology.orchestra.run.smithi173.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 198 kB/s | 25 kB 00:00 2023-12-12T09:45:47.964 INFO:teuthology.orchestra.run.smithi111.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.5 MB/s | 94 kB 00:00 2023-12-12T09:45:47.980 INFO:teuthology.orchestra.run.smithi173.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 271 kB/s | 41 kB 00:00 2023-12-12T09:45:47.989 INFO:teuthology.orchestra.run.smithi111.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 292 kB/s | 25 kB 00:00 2023-12-12T09:45:48.005 INFO:teuthology.orchestra.run.smithi173.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 639 kB/s | 49 kB 00:00 2023-12-12T09:45:48.014 INFO:teuthology.orchestra.run.smithi111.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 370 kB/s | 41 kB 00:00 2023-12-12T09:45:48.038 INFO:teuthology.orchestra.run.smithi173.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-12-12T09:45:48.039 INFO:teuthology.orchestra.run.smithi111.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-12-12T09:45:48.055 INFO:teuthology.orchestra.run.smithi173.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.9 MB/s | 149 kB 00:00 2023-12-12T09:45:48.073 INFO:teuthology.orchestra.run.smithi111.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-12-12T09:45:48.081 INFO:teuthology.orchestra.run.smithi173.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 5.0 MB/s | 213 kB 00:00 2023-12-12T09:45:48.090 INFO:teuthology.orchestra.run.smithi111.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.9 MB/s | 149 kB 00:00 2023-12-12T09:45:48.104 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2023-12-12T09:45:48.115 INFO:teuthology.orchestra.run.smithi111.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 5.0 MB/s | 213 kB 00:00 2023-12-12T09:45:48.173 INFO:teuthology.orchestra.run.smithi173.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 24 MB/s | 2.2 MB 00:00 2023-12-12T09:45:48.199 INFO:teuthology.orchestra.run.smithi111.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 26 MB/s | 2.2 MB 00:00 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================================================ 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================================================ 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout:Installing group/module packages: 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-12-12T09:45:48.205 INFO:teuthology.orchestra.run.smithi042.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:Installing module profiles: 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout: container-tools/common 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling module streams: 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout: container-tools rhel8 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:============================================================================================================ 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout:Install 10 Packages 2023-12-12T09:45:48.206 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:45:48.207 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 18 M 2023-12-12T09:45:48.207 INFO:teuthology.orchestra.run.smithi042.stdout:Installed size: 65 M 2023-12-12T09:45:48.207 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2023-12-12T09:45:48.276 INFO:teuthology.orchestra.run.smithi042.stdout:(1/10): yajl-2.1.0-10.el8.x86_64.rpm 601 kB/s | 41 kB 00:00 2023-12-12T09:45:48.301 INFO:teuthology.orchestra.run.smithi042.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 267 kB/s | 25 kB 00:00 2023-12-12T09:45:48.306 INFO:teuthology.orchestra.run.smithi173.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-12-12T09:45:48.326 INFO:teuthology.orchestra.run.smithi042.stdout:(3/10): udica-0.2.6-4.module+el8.6.0+16771+28df 988 kB/s | 49 kB 00:00 2023-12-12T09:45:48.341 INFO:teuthology.orchestra.run.smithi111.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-12-12T09:45:48.343 INFO:teuthology.orchestra.run.smithi042.stdout:(4/10): python3-pyxdg-0.25-16.el8.noarch.rpm 693 kB/s | 94 kB 00:00 2023-12-12T09:45:48.377 INFO:teuthology.orchestra.run.smithi042.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.8 MB/s | 524 kB 00:00 2023-12-12T09:45:48.402 INFO:teuthology.orchestra.run.smithi042.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-12-12T09:45:48.406 INFO:teuthology.orchestra.run.smithi173.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-12-12T09:45:48.407 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:45:48.407 INFO:teuthology.orchestra.run.smithi173.stdout:Total 32 MB/s | 18 MB 00:00 2023-12-12T09:45:48.425 INFO:teuthology.orchestra.run.smithi111.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-12-12T09:45:48.425 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:45:48.425 INFO:teuthology.orchestra.run.smithi111.stdout:Total 35 MB/s | 18 MB 00:00 2023-12-12T09:45:48.427 INFO:teuthology.orchestra.run.smithi042.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-12-12T09:45:48.528 INFO:teuthology.orchestra.run.smithi042.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-12-12T09:45:48.607 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2023-12-12T09:45:48.626 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2023-12-12T09:45:48.626 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2023-12-12T09:45:48.627 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2023-12-12T09:45:48.645 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2023-12-12T09:45:48.646 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2023-12-12T09:45:48.653 INFO:teuthology.orchestra.run.smithi042.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 28 MB/s | 7.0 MB 00:00 2023-12-12T09:45:48.753 INFO:teuthology.orchestra.run.smithi042.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 19 MB/s | 8.0 MB 00:00 2023-12-12T09:45:48.753 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:45:48.753 INFO:teuthology.orchestra.run.smithi042.stdout:Total 33 MB/s | 18 MB 00:00 2023-12-12T09:45:48.807 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2023-12-12T09:45:48.811 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2023-12-12T09:45:48.826 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2023-12-12T09:45:48.830 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2023-12-12T09:45:48.958 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2023-12-12T09:45:48.979 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2023-12-12T09:45:48.979 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2023-12-12T09:45:49.175 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2023-12-12T09:45:49.179 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2023-12-12T09:45:49.307 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2023-12-12T09:45:49.342 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2023-12-12T09:45:49.507 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-12T09:45:49.526 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-12T09:45:49.698 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-12T09:45:49.702 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-12T09:45:49.725 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2023-12-12T09:45:49.897 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-12T09:45:49.917 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-12T09:45:49.949 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-12-12T09:45:50.125 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-12T09:45:50.136 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-12T09:45:50.331 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-12-12T09:45:50.354 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-12T09:45:50.512 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-12-12T09:45:50.535 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-12T09:45:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-12T09:45:50.916 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-12-12T09:45:51.356 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-12T09:45:51.502 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-12T09:45:51.728 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-12-12T09:45:52.226 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-12T09:45:52.360 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-12T09:45:52.437 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-12T09:45:52.591 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-12T09:45:52.621 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-12T09:45:52.640 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-12-12T09:45:52.768 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-12T09:45:52.816 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-12T09:45:52.849 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-12-12T09:45:52.979 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-12T09:45:53.048 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-12-12T09:45:53.084 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-12T09:45:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-12T09:45:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-12T09:45:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-12T09:45:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-12T09:45:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-12T09:45:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-12T09:45:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-12T09:45:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-12T09:45:53.085 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-12T09:45:53.208 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-12T09:45:53.240 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-12T09:45:53.240 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-12T09:45:53.240 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-12T09:45:53.240 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-12T09:45:53.240 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-12T09:45:53.240 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-12T09:45:53.240 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-12T09:45:53.241 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-12T09:45:53.241 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-12T09:45:53.241 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-12T09:45:53.468 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-12-12T09:45:53.469 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-12-12T09:45:53.469 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-12-12T09:45:53.469 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-12-12T09:45:53.469 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-12-12T09:45:53.469 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-12-12T09:45:53.469 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-12-12T09:45:53.469 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-12-12T09:45:53.469 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-12-12T09:45:53.469 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-12-12T09:45:57.291 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-12T09:45:57.291 INFO:teuthology.orchestra.run.smithi111.stdout:Installed products updated. 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-12T09:45:57.491 INFO:teuthology.orchestra.run.smithi111.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-12T09:45:57.492 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:45:57.492 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-12-12T09:45:57.626 DEBUG:teuthology.parallel:result is None 2023-12-12T09:45:57.734 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-12T09:45:57.734 INFO:teuthology.orchestra.run.smithi173.stdout:Installed products updated. 2023-12-12T09:45:57.917 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-12-12T09:45:57.917 INFO:teuthology.orchestra.run.smithi042.stdout:Installed products updated. 2023-12-12T09:45:57.932 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:45:57.932 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2023-12-12T09:45:57.932 INFO:teuthology.orchestra.run.smithi173.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:57.932 INFO:teuthology.orchestra.run.smithi173.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:45:57.933 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2023-12-12T09:45:58.067 DEBUG:teuthology.parallel:result is None 2023-12-12T09:45:58.130 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:45:58.130 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2023-12-12T09:45:58.130 INFO:teuthology.orchestra.run.smithi042.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:58.130 INFO:teuthology.orchestra.run.smithi042.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-12-12T09:45:58.130 INFO:teuthology.orchestra.run.smithi042.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:58.130 INFO:teuthology.orchestra.run.smithi042.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-12-12T09:45:58.130 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-12T09:45:58.131 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-12T09:45:58.131 INFO:teuthology.orchestra.run.smithi042.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:58.131 INFO:teuthology.orchestra.run.smithi042.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-12-12T09:45:58.131 INFO:teuthology.orchestra.run.smithi042.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-12-12T09:45:58.131 INFO:teuthology.orchestra.run.smithi042.stdout: yajl-2.1.0-10.el8.x86_64 2023-12-12T09:45:58.131 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:45:58.131 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2023-12-12T09:45:58.272 DEBUG:teuthology.parallel:result is None 2023-12-12T09:45:58.272 INFO:teuthology.run_tasks:Running task install... 2023-12-12T09:45:58.282 DEBUG:teuthology.task.install:project ceph 2023-12-12T09:45:58.282 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9'}} 2023-12-12T09:45:58.282 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9'} 2023-12-12T09:45:58.282 INFO:teuthology.task.install:Using flavor: default 2023-12-12T09:45:58.289 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-12T09:45:58.289 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-12T09:45:58.289 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'tag': None, 'wait_for_package': False} 2023-12-12T09:45:58.290 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:45:58.292 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'tag': None, 'wait_for_package': False} 2023-12-12T09:45:58.292 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:45:58.293 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'tag': None, 'wait_for_package': False} 2023-12-12T09:45:58.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:45:58.435 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/ 2023-12-12T09:45:58.435 INFO:teuthology.task.install.rpm:Package version is 19.0.0-96.gf1fbf3c5 2023-12-12T09:45:58.445 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/ 2023-12-12T09:45:58.446 INFO:teuthology.task.install.rpm:Package version is 19.0.0-96.gf1fbf3c5 2023-12-12T09:45:58.451 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/ 2023-12-12T09:45:58.451 INFO:teuthology.task.install.rpm:Package version is 19.0.0-96.gf1fbf3c5 2023-12-12T09:45:58.614 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-12T09:45:58.615 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:45:58.615 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-12T09:45:58.625 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-12T09:45:58.626 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:45:58.626 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-12T09:45:58.629 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-12T09:45:58.629 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:45:58.629 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-12T09:45:58.644 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-12T09:45:58.644 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-12T09:45:58.652 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-12T09:45:58.653 DEBUG:teuthology.orchestra.run.smithi173:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-12T09:45:58.662 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-12T09:45:58.662 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-12T09:45:58.714 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-12T09:45:58.720 DEBUG:teuthology.orchestra.run.smithi173:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-12T09:45:58.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-12T09:45:58.794 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-12T09:45:58.796 DEBUG:teuthology.orchestra.run.smithi173:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-12T09:45:58.817 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-12T09:45:58.861 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2023-12-12T09:45:58.863 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2023-12-12T09:45:58.864 INFO:teuthology.orchestra.run.smithi173.stdout:check_obsoletes = 1 2023-12-12T09:45:58.864 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum clean all 2023-12-12T09:45:58.886 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2023-12-12T09:45:58.887 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2023-12-12T09:45:59.237 INFO:teuthology.orchestra.run.smithi173.stdout:Updating Subscription Management repositories. 2023-12-12T09:45:59.254 INFO:teuthology.orchestra.run.smithi042.stdout:Updating Subscription Management repositories. 2023-12-12T09:45:59.264 INFO:teuthology.orchestra.run.smithi111.stdout:Updating Subscription Management repositories. 2023-12-12T09:45:59.667 INFO:teuthology.orchestra.run.smithi173.stdout:44 files removed 2023-12-12T09:45:59.677 INFO:teuthology.orchestra.run.smithi111.stdout:44 files removed 2023-12-12T09:45:59.697 INFO:teuthology.orchestra.run.smithi042.stdout:44 files removed 2023-12-12T09:45:59.715 DEBUG:teuthology.orchestra.run.smithi173:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-12T09:45:59.723 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-12T09:45:59.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-12T09:46:00.047 INFO:teuthology.orchestra.run.smithi173.stdout:Updating Subscription Management repositories. 2023-12-12T09:46:00.053 INFO:teuthology.orchestra.run.smithi111.stdout:Updating Subscription Management repositories. 2023-12-12T09:46:00.097 INFO:teuthology.orchestra.run.smithi042.stdout:Updating Subscription Management repositories. 2023-12-12T09:46:01.018 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 311 kB/s | 81 kB 00:00 2023-12-12T09:46:01.020 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 308 kB/s | 81 kB 00:00 2023-12-12T09:46:01.169 INFO:teuthology.orchestra.run.smithi173.stdout:ceph packages for x86_64 181 kB/s | 81 kB 00:00 2023-12-12T09:46:01.321 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 35 kB/s | 9.5 kB 00:00 2023-12-12T09:46:01.394 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 28 kB/s | 9.5 kB 00:00 2023-12-12T09:46:01.436 INFO:teuthology.orchestra.run.smithi173.stdout:ceph noarch packages 40 kB/s | 9.5 kB 00:00 2023-12-12T09:46:01.638 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 5.0 kB/s | 1.5 kB 00:00 2023-12-12T09:46:01.711 INFO:teuthology.orchestra.run.smithi173.stdout:ceph source packages 5.8 kB/s | 1.5 kB 00:00 2023-12-12T09:46:01.770 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 4.2 kB/s | 1.5 kB 00:00 2023-12-12T09:46:02.737 INFO:teuthology.orchestra.run.smithi173.stdout:Copr repo for python3-asyncssh owned by ceph 3.4 kB/s | 3.5 kB 00:01 2023-12-12T09:46:02.804 INFO:teuthology.orchestra.run.smithi042.stdout:Copr repo for python3-asyncssh owned by ceph 3.4 kB/s | 3.5 kB 00:01 2023-12-12T09:46:02.831 INFO:teuthology.orchestra.run.smithi111.stdout:Copr repo for python3-asyncssh owned by ceph 3.0 kB/s | 3.5 kB 00:01 2023-12-12T09:46:03.913 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-12-12T09:46:04.338 INFO:teuthology.orchestra.run.smithi173.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-12-12T09:46:04.425 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-12-12T09:46:07.713 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 474 kB/s | 24 kB 00:00 2023-12-12T09:46:07.907 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 378 kB/s | 24 kB 00:00 2023-12-12T09:46:07.913 INFO:teuthology.orchestra.run.smithi173.stdout:lab-extras 420 kB/s | 24 kB 00:00 2023-12-12T09:46:08.657 INFO:teuthology.orchestra.run.smithi042.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-12-12T09:46:08.900 INFO:teuthology.orchestra.run.smithi111.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-12-12T09:46:08.931 INFO:teuthology.orchestra.run.smithi173.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 MB/s | 53 MB 00:00 2023-12-12T09:46:17.060 INFO:teuthology.orchestra.run.smithi111.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-12-12T09:46:17.199 INFO:teuthology.orchestra.run.smithi173.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-12-12T09:46:17.258 INFO:teuthology.orchestra.run.smithi042.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 54 MB/s | 53 MB 00:00 2023-12-12T09:46:28.025 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-12T09:46:28.026 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-12T09:46:28.040 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-12T09:46:28.042 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-12T09:46:28.043 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-12T09:46:28.044 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-12T09:46:28.047 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-12T09:46:28.050 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-12T09:46:28.051 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-12T09:46:28.063 INFO:teuthology.orchestra.run.smithi173.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-12T09:46:28.064 INFO:teuthology.orchestra.run.smithi173.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-12T09:46:28.077 INFO:teuthology.orchestra.run.smithi173.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-12T09:46:28.078 INFO:teuthology.orchestra.run.smithi173.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-12T09:46:28.079 INFO:teuthology.orchestra.run.smithi173.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-12T09:46:28.080 INFO:teuthology.orchestra.run.smithi173.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-12T09:46:28.083 INFO:teuthology.orchestra.run.smithi173.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-12T09:46:28.086 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-12T09:46:28.087 INFO:teuthology.orchestra.run.smithi173.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-12T09:46:28.181 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-12-12T09:46:28.193 INFO:teuthology.orchestra.run.smithi111.stdout:=========================================================================================================================== 2023-12-12T09:46:28.193 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2023-12-12T09:46:28.193 INFO:teuthology.orchestra.run.smithi111.stdout:=========================================================================================================================== 2023-12-12T09:46:28.193 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2023-12-12T09:46:28.193 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 6.4 k 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 5.4 M 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 919 k 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 171 k 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 1.6 M 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 133 k 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 1.8 M 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 7.4 M 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 50 k 2023-12-12T09:46:28.194 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 14 M 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 49 M 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 262 k 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 416 k 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 263 k 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 18 k 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 31 k 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 838 k 2023-12-12T09:46:28.195 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 139 k 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 197 k 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 371 k 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 364 k 2023-12-12T09:46:28.196 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 106 k 2023-12-12T09:46:28.197 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 93 k 2023-12-12T09:46:28.197 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 3.6 M 2023-12-12T09:46:28.198 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 181 k 2023-12-12T09:46:28.198 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-12T09:46:28.198 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2023-12-12T09:46:28.198 INFO:teuthology.orchestra.run.smithi111.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-12T09:46:28.198 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 3.8 M 2023-12-12T09:46:28.198 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 3.7 M 2023-12-12T09:46:28.198 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 23 M 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 28 k 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 2.3 M 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 243 k 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 4.6 M 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 18 M 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 15 k 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 24 k 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-12T09:46:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 183 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 517 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 6.9 M 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-12T09:46:28.200 INFO:teuthology.orchestra.run.smithi111.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-12T09:46:28.201 INFO:teuthology.orchestra.run.smithi111.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 45 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 128 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-12T09:46:28.202 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-12T09:46:28.203 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-12T09:46:28.204 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout:Enabling module streams: 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql 10 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2023-12-12T09:46:28.205 INFO:teuthology.orchestra.run.smithi111.stdout:=========================================================================================================================== 2023-12-12T09:46:28.206 INFO:teuthology.orchestra.run.smithi111.stdout:Install 129 Packages 2023-12-12T09:46:28.206 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 3 Packages 2023-12-12T09:46:28.206 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:46:28.206 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 209 M 2023-12-12T09:46:28.206 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2023-12-12T09:46:28.217 INFO:teuthology.orchestra.run.smithi173.stdout:Dependencies resolved. 2023-12-12T09:46:28.230 INFO:teuthology.orchestra.run.smithi173.stdout:=========================================================================================================================== 2023-12-12T09:46:28.230 INFO:teuthology.orchestra.run.smithi173.stdout: Package Arch Version Repository Size 2023-12-12T09:46:28.230 INFO:teuthology.orchestra.run.smithi173.stdout:=========================================================================================================================== 2023-12-12T09:46:28.230 INFO:teuthology.orchestra.run.smithi173.stdout:Installing: 2023-12-12T09:46:28.230 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-12T09:46:28.230 INFO:teuthology.orchestra.run.smithi173.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-12T09:46:28.230 INFO:teuthology.orchestra.run.smithi173.stdout: ceph x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 6.4 k 2023-12-12T09:46:28.230 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 5.4 M 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 919 k 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 171 k 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 1.6 M 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 133 k 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 1.8 M 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 7.4 M 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 50 k 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 14 M 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 49 M 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 262 k 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 416 k 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 263 k 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 18 k 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-12T09:46:28.231 INFO:teuthology.orchestra.run.smithi173.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 31 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 838 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 139 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 197 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 371 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 364 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 106 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 93 k 2023-12-12T09:46:28.232 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 3.6 M 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 181 k 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrading: 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 3.8 M 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 3.7 M 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout:Installing dependencies: 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 23 M 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 28 k 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 2.3 M 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 243 k 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 4.6 M 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 18 M 2023-12-12T09:46:28.233 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 15 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 24 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 183 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-12T09:46:28.234 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 517 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 6.9 M 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-12T09:46:28.235 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 45 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 128 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-12T09:46:28.236 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-12T09:46:28.237 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-12T09:46:28.238 INFO:teuthology.orchestra.run.smithi173.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout:Enabling module streams: 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql 10 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction Summary 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout:=========================================================================================================================== 2023-12-12T09:46:28.239 INFO:teuthology.orchestra.run.smithi173.stdout:Install 129 Packages 2023-12-12T09:46:28.240 INFO:teuthology.orchestra.run.smithi173.stdout:Upgrade 3 Packages 2023-12-12T09:46:28.240 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:46:28.240 INFO:teuthology.orchestra.run.smithi173.stdout:Total download size: 209 M 2023-12-12T09:46:28.240 INFO:teuthology.orchestra.run.smithi173.stdout:Downloading Packages: 2023-12-12T09:46:28.494 INFO:teuthology.orchestra.run.smithi111.stdout:(1/132): ceph-19.0.0-96.gf1fbf3c5.el8.x86_64.rp 56 kB/s | 6.4 kB 00:00 2023-12-12T09:46:28.599 INFO:teuthology.orchestra.run.smithi173.stdout:(1/132): ceph-19.0.0-96.gf1fbf3c5.el8.x86_64.rp 35 kB/s | 6.4 kB 00:00 2023-12-12T09:46:28.719 INFO:teuthology.orchestra.run.smithi111.stdout:(2/132): ceph-fuse-19.0.0-96.gf1fbf3c5.el8.x86_ 4.0 MB/s | 919 kB 00:00 2023-12-12T09:46:28.761 INFO:teuthology.orchestra.run.smithi111.stdout:(3/132): ceph-immutable-object-cache-19.0.0-96. 4.0 MB/s | 171 kB 00:00 2023-12-12T09:46:29.016 INFO:teuthology.orchestra.run.smithi173.stdout:(2/132): ceph-fuse-19.0.0-96.gf1fbf3c5.el8.x86_ 2.2 MB/s | 919 kB 00:00 2023-12-12T09:46:29.100 INFO:teuthology.orchestra.run.smithi173.stdout:(3/132): ceph-immutable-object-cache-19.0.0-96. 2.0 MB/s | 171 kB 00:00 2023-12-12T09:46:29.170 INFO:teuthology.orchestra.run.smithi111.stdout:(4/132): ceph-mds-19.0.0-96.gf1fbf3c5.el8.x86_6 5.7 MB/s | 2.3 MB 00:00 2023-12-12T09:46:29.316 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-12T09:46:29.317 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-12T09:46:29.332 INFO:teuthology.orchestra.run.smithi042.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-12T09:46:29.333 INFO:teuthology.orchestra.run.smithi042.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-12T09:46:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-12T09:46:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-12T09:46:29.338 INFO:teuthology.orchestra.run.smithi042.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-12T09:46:29.341 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-12T09:46:29.343 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-12T09:46:29.489 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2023-12-12T09:46:29.502 INFO:teuthology.orchestra.run.smithi042.stdout:=========================================================================================================================== 2023-12-12T09:46:29.502 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2023-12-12T09:46:29.502 INFO:teuthology.orchestra.run.smithi042.stdout:=========================================================================================================================== 2023-12-12T09:46:29.502 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2023-12-12T09:46:29.502 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-12-12T09:46:29.502 INFO:teuthology.orchestra.run.smithi042.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 6.4 k 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 5.4 M 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 919 k 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 171 k 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 1.6 M 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 133 k 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 1.8 M 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 7.4 M 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 50 k 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 14 M 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 49 M 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 262 k 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 416 k 2023-12-12T09:46:29.503 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 263 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 18 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 31 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 838 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 139 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-12T09:46:29.504 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 197 k 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 371 k 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 364 k 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 106 k 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 93 k 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 3.6 M 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 181 k 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 3.8 M 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 3.7 M 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 23 M 2023-12-12T09:46:29.505 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 28 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 2.3 M 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 243 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 4.6 M 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 18 M 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:19.0.0-96.gf1fbf3c5.el8 ceph-noarch 15 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 24 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 183 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-12T09:46:29.506 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 517 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 6.9 M 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks noarch 3.5.0-1.el8 epel 167 k 2023-12-12T09:46:29.507 INFO:teuthology.orchestra.run.smithi042.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-12T09:46:29.508 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 45 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:19.0.0-96.gf1fbf3c5.el8 ceph 128 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-12T09:46:29.509 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-12T09:46:29.510 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-12T09:46:29.511 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling module streams: 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql 10 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout:=========================================================================================================================== 2023-12-12T09:46:29.512 INFO:teuthology.orchestra.run.smithi042.stdout:Install 129 Packages 2023-12-12T09:46:29.513 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 3 Packages 2023-12-12T09:46:29.513 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:46:29.513 INFO:teuthology.orchestra.run.smithi111.stdout:(5/132): ceph-mgr-19.0.0-96.gf1fbf3c5.el8.x86_6 4.9 MB/s | 1.6 MB 00:00 2023-12-12T09:46:29.513 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 209 M 2023-12-12T09:46:29.514 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2023-12-12T09:46:29.858 INFO:teuthology.orchestra.run.smithi173.stdout:(4/132): ceph-mds-19.0.0-96.gf1fbf3c5.el8.x86_6 3.0 MB/s | 2.3 MB 00:00 2023-12-12T09:46:29.920 INFO:teuthology.orchestra.run.smithi111.stdout:(6/132): ceph-base-19.0.0-96.gf1fbf3c5.el8.x86_ 3.5 MB/s | 5.4 MB 00:01 2023-12-12T09:46:30.007 INFO:teuthology.orchestra.run.smithi042.stdout:(1/132): ceph-19.0.0-96.gf1fbf3c5.el8.x86_64.rp 18 kB/s | 6.4 kB 00:00 2023-12-12T09:46:30.025 INFO:teuthology.orchestra.run.smithi173.stdout:(5/132): ceph-base-19.0.0-96.gf1fbf3c5.el8.x86_ 3.3 MB/s | 5.4 MB 00:01 2023-12-12T09:46:30.409 INFO:teuthology.orchestra.run.smithi173.stdout:(6/132): ceph-mgr-19.0.0-96.gf1fbf3c5.el8.x86_6 3.0 MB/s | 1.6 MB 00:00 2023-12-12T09:46:30.445 INFO:teuthology.orchestra.run.smithi111.stdout:(7/132): ceph-mon-19.0.0-96.gf1fbf3c5.el8.x86_6 4.9 MB/s | 4.6 MB 00:00 2023-12-12T09:46:30.567 INFO:teuthology.orchestra.run.smithi042.stdout:(2/132): ceph-fuse-19.0.0-96.gf1fbf3c5.el8.x86_ 1.6 MB/s | 919 kB 00:00 2023-12-12T09:46:30.668 INFO:teuthology.orchestra.run.smithi042.stdout:(3/132): ceph-immutable-object-cache-19.0.0-96. 1.7 MB/s | 171 kB 00:00 2023-12-12T09:46:31.418 INFO:teuthology.orchestra.run.smithi042.stdout:(4/132): ceph-mds-19.0.0-96.gf1fbf3c5.el8.x86_6 3.1 MB/s | 2.3 MB 00:00 2023-12-12T09:46:31.601 INFO:teuthology.orchestra.run.smithi173.stdout:(7/132): ceph-mon-19.0.0-96.gf1fbf3c5.el8.x86_6 2.9 MB/s | 4.6 MB 00:01 2023-12-12T09:46:31.877 INFO:teuthology.orchestra.run.smithi042.stdout:(5/132): ceph-mgr-19.0.0-96.gf1fbf3c5.el8.x86_6 3.5 MB/s | 1.6 MB 00:00 2023-12-12T09:46:32.310 INFO:teuthology.orchestra.run.smithi173.stdout:(8/132): ceph-common-19.0.0-96.gf1fbf3c5.el8.x8 5.8 MB/s | 23 MB 00:03 2023-12-12T09:46:32.401 INFO:teuthology.orchestra.run.smithi173.stdout:(9/132): ceph-selinux-19.0.0-96.gf1fbf3c5.el8.x 266 kB/s | 24 kB 00:00 2023-12-12T09:46:32.694 INFO:teuthology.orchestra.run.smithi042.stdout:(6/132): ceph-mon-19.0.0-96.gf1fbf3c5.el8.x86_6 5.7 MB/s | 4.6 MB 00:00 2023-12-12T09:46:33.795 INFO:teuthology.orchestra.run.smithi111.stdout:(8/132): ceph-osd-19.0.0-96.gf1fbf3c5.el8.x86_6 4.6 MB/s | 18 MB 00:03 2023-12-12T09:46:33.929 INFO:teuthology.orchestra.run.smithi111.stdout:(9/132): ceph-radosgw-19.0.0-96.gf1fbf3c5.el8.x 4.0 MB/s | 14 MB 00:03 2023-12-12T09:46:33.954 INFO:teuthology.orchestra.run.smithi111.stdout:(10/132): ceph-selinux-19.0.0-96.gf1fbf3c5.el8. 153 kB/s | 24 kB 00:00 2023-12-12T09:46:34.146 INFO:teuthology.orchestra.run.smithi111.stdout:(11/132): cephfs-mirror-19.0.0-96.gf1fbf3c5.el8 1.3 MB/s | 263 kB 00:00 2023-12-12T09:46:34.263 INFO:teuthology.orchestra.run.smithi111.stdout:(12/132): libcephfs-devel-19.0.0-96.gf1fbf3c5.e 266 kB/s | 31 kB 00:00 2023-12-12T09:46:34.639 INFO:teuthology.orchestra.run.smithi111.stdout:(13/132): libcephfs2-19.0.0-96.gf1fbf3c5.el8.x8 2.2 MB/s | 838 kB 00:00 2023-12-12T09:46:34.756 INFO:teuthology.orchestra.run.smithi111.stdout:(14/132): libcephsqlite-19.0.0-96.gf1fbf3c5.el8 1.5 MB/s | 183 kB 00:00 2023-12-12T09:46:34.872 INFO:teuthology.orchestra.run.smithi111.stdout:(15/132): librados-devel-19.0.0-96.gf1fbf3c5.el 1.2 MB/s | 139 kB 00:00 2023-12-12T09:46:34.977 INFO:teuthology.orchestra.run.smithi042.stdout:(7/132): ceph-base-19.0.0-96.gf1fbf3c5.el8.x86_ 1.0 MB/s | 5.4 MB 00:05 2023-12-12T09:46:35.064 INFO:teuthology.orchestra.run.smithi111.stdout:(16/132): libradosstriper1-19.0.0-96.gf1fbf3c5. 2.6 MB/s | 517 kB 00:00 2023-12-12T09:46:35.543 INFO:teuthology.orchestra.run.smithi173.stdout:(10/132): ceph-osd-19.0.0-96.gf1fbf3c5.el8.x86_ 3.5 MB/s | 18 MB 00:05 2023-12-12T09:46:35.582 INFO:teuthology.orchestra.run.smithi111.stdout:(17/132): ceph-common-19.0.0-96.gf1fbf3c5.el8.x 3.1 MB/s | 23 MB 00:07 2023-12-12T09:46:35.660 INFO:teuthology.orchestra.run.smithi173.stdout:(11/132): cephfs-mirror-19.0.0-96.gf1fbf3c5.el8 2.2 MB/s | 263 kB 00:00 2023-12-12T09:46:35.698 INFO:teuthology.orchestra.run.smithi111.stdout:(18/132): python3-ceph-argparse-19.0.0-96.gf1fb 384 kB/s | 45 kB 00:00 2023-12-12T09:46:35.785 INFO:teuthology.orchestra.run.smithi173.stdout:(12/132): libcephfs-devel-19.0.0-96.gf1fbf3c5.e 248 kB/s | 31 kB 00:00 2023-12-12T09:46:35.924 INFO:teuthology.orchestra.run.smithi111.stdout:(19/132): python3-ceph-common-19.0.0-96.gf1fbf3 571 kB/s | 128 kB 00:00 2023-12-12T09:46:36.127 INFO:teuthology.orchestra.run.smithi173.stdout:(13/132): libcephfs2-19.0.0-96.gf1fbf3c5.el8.x8 2.4 MB/s | 838 kB 00:00 2023-12-12T09:46:36.136 INFO:teuthology.orchestra.run.smithi111.stdout:(20/132): python3-cephfs-19.0.0-96.gf1fbf3c5.el 928 kB/s | 197 kB 00:00 2023-12-12T09:46:36.178 INFO:teuthology.orchestra.run.smithi042.stdout:(8/132): ceph-osd-19.0.0-96.gf1fbf3c5.el8.x86_6 5.2 MB/s | 18 MB 00:03 2023-12-12T09:46:36.229 INFO:teuthology.orchestra.run.smithi042.stdout:(9/132): ceph-selinux-19.0.0-96.gf1fbf3c5.el8.x 486 kB/s | 24 kB 00:00 2023-12-12T09:46:36.229 INFO:teuthology.orchestra.run.smithi173.stdout:(14/132): libcephsqlite-19.0.0-96.gf1fbf3c5.el8 1.8 MB/s | 183 kB 00:00 2023-12-12T09:46:36.316 INFO:teuthology.orchestra.run.smithi111.stdout:(21/132): python3-rados-19.0.0-96.gf1fbf3c5.el8 2.0 MB/s | 371 kB 00:00 2023-12-12T09:46:36.395 INFO:teuthology.orchestra.run.smithi173.stdout:(15/132): ceph-radosgw-19.0.0-96.gf1fbf3c5.el8. 2.9 MB/s | 14 MB 00:04 2023-12-12T09:46:36.420 INFO:teuthology.orchestra.run.smithi173.stdout:(16/132): librados-devel-19.0.0-96.gf1fbf3c5.el 725 kB/s | 139 kB 00:00 2023-12-12T09:46:36.470 INFO:teuthology.orchestra.run.smithi042.stdout:(10/132): ceph-common-19.0.0-96.gf1fbf3c5.el8.x 3.3 MB/s | 23 MB 00:06 2023-12-12T09:46:36.508 INFO:teuthology.orchestra.run.smithi111.stdout:(22/132): python3-rbd-19.0.0-96.gf1fbf3c5.el8.x 1.9 MB/s | 364 kB 00:00 2023-12-12T09:46:36.570 INFO:teuthology.orchestra.run.smithi042.stdout:(11/132): cephfs-mirror-19.0.0-96.gf1fbf3c5.el8 2.6 MB/s | 263 kB 00:00 2023-12-12T09:46:36.633 INFO:teuthology.orchestra.run.smithi111.stdout:(23/132): python3-rgw-19.0.0-96.gf1fbf3c5.el8.x 852 kB/s | 106 kB 00:00 2023-12-12T09:46:36.654 INFO:teuthology.orchestra.run.smithi173.stdout:(17/132): libradosstriper1-19.0.0-96.gf1fbf3c5. 2.0 MB/s | 517 kB 00:00 2023-12-12T09:46:36.662 INFO:teuthology.orchestra.run.smithi042.stdout:(12/132): libcephfs-devel-19.0.0-96.gf1fbf3c5.e 339 kB/s | 31 kB 00:00 2023-12-12T09:46:36.733 INFO:teuthology.orchestra.run.smithi111.stdout:(24/132): rbd-fuse-19.0.0-96.gf1fbf3c5.el8.x86_ 936 kB/s | 93 kB 00:00 2023-12-12T09:46:36.754 INFO:teuthology.orchestra.run.smithi173.stdout:(18/132): python3-ceph-argparse-19.0.0-96.gf1fb 447 kB/s | 45 kB 00:00 2023-12-12T09:46:36.871 INFO:teuthology.orchestra.run.smithi173.stdout:(19/132): python3-ceph-common-19.0.0-96.gf1fbf3 1.1 MB/s | 128 kB 00:00 2023-12-12T09:46:36.871 INFO:teuthology.orchestra.run.smithi042.stdout:(13/132): libcephfs2-19.0.0-96.gf1fbf3c5.el8.x8 3.9 MB/s | 838 kB 00:00 2023-12-12T09:46:36.971 INFO:teuthology.orchestra.run.smithi042.stdout:(14/132): libcephsqlite-19.0.0-96.gf1fbf3c5.el8 1.8 MB/s | 183 kB 00:00 2023-12-12T09:46:36.979 INFO:teuthology.orchestra.run.smithi173.stdout:(20/132): python3-cephfs-19.0.0-96.gf1fbf3c5.el 1.8 MB/s | 197 kB 00:00 2023-12-12T09:46:37.072 INFO:teuthology.orchestra.run.smithi042.stdout:(15/132): librados-devel-19.0.0-96.gf1fbf3c5.el 1.4 MB/s | 139 kB 00:00 2023-12-12T09:46:37.197 INFO:teuthology.orchestra.run.smithi042.stdout:(16/132): libradosstriper1-19.0.0-96.gf1fbf3c5. 4.0 MB/s | 517 kB 00:00 2023-12-12T09:46:37.217 INFO:teuthology.orchestra.run.smithi111.stdout:(25/132): librgw2-19.0.0-96.gf1fbf3c5.el8.x86_6 3.2 MB/s | 6.9 MB 00:02 2023-12-12T09:46:37.239 INFO:teuthology.orchestra.run.smithi173.stdout:(21/132): python3-rados-19.0.0-96.gf1fbf3c5.el8 1.4 MB/s | 371 kB 00:00 2023-12-12T09:46:37.334 INFO:teuthology.orchestra.run.smithi111.stdout:(26/132): rbd-nbd-19.0.0-96.gf1fbf3c5.el8.x86_6 1.5 MB/s | 181 kB 00:00 2023-12-12T09:46:37.451 INFO:teuthology.orchestra.run.smithi111.stdout:(27/132): ceph-grafana-dashboards-19.0.0-96.gf1 240 kB/s | 28 kB 00:00 2023-12-12T09:46:37.505 INFO:teuthology.orchestra.run.smithi173.stdout:(22/132): python3-rbd-19.0.0-96.gf1fbf3c5.el8.x 1.3 MB/s | 364 kB 00:00 2023-12-12T09:46:37.577 INFO:teuthology.orchestra.run.smithi111.stdout:(28/132): ceph-mgr-cephadm-19.0.0-96.gf1fbf3c5. 1.0 MB/s | 133 kB 00:00 2023-12-12T09:46:37.614 INFO:teuthology.orchestra.run.smithi173.stdout:(23/132): python3-rgw-19.0.0-96.gf1fbf3c5.el8.x 984 kB/s | 106 kB 00:00 2023-12-12T09:46:37.731 INFO:teuthology.orchestra.run.smithi173.stdout:(24/132): rbd-fuse-19.0.0-96.gf1fbf3c5.el8.x86_ 801 kB/s | 93 kB 00:00 2023-12-12T09:46:38.160 INFO:teuthology.orchestra.run.smithi111.stdout:(29/132): ceph-mgr-dashboard-19.0.0-96.gf1fbf3c 3.0 MB/s | 1.8 MB 00:00 2023-12-12T09:46:38.214 INFO:teuthology.orchestra.run.smithi173.stdout:(25/132): librgw2-19.0.0-96.gf1fbf3c5.el8.x86_6 3.9 MB/s | 6.9 MB 00:01 2023-12-12T09:46:38.323 INFO:teuthology.orchestra.run.smithi173.stdout:(26/132): rbd-nbd-19.0.0-96.gf1fbf3c5.el8.x86_6 1.6 MB/s | 181 kB 00:00 2023-12-12T09:46:38.423 INFO:teuthology.orchestra.run.smithi173.stdout:(27/132): ceph-grafana-dashboards-19.0.0-96.gf1 280 kB/s | 28 kB 00:00 2023-12-12T09:46:38.532 INFO:teuthology.orchestra.run.smithi173.stdout:(28/132): ceph-mgr-cephadm-19.0.0-96.gf1fbf3c5. 1.2 MB/s | 133 kB 00:00 2023-12-12T09:46:38.655 INFO:teuthology.orchestra.run.smithi042.stdout:(17/132): librgw2-19.0.0-96.gf1fbf3c5.el8.x86_6 4.7 MB/s | 6.9 MB 00:01 2023-12-12T09:46:38.677 INFO:teuthology.orchestra.run.smithi111.stdout:(30/132): rbd-mirror-19.0.0-96.gf1fbf3c5.el8.x8 1.8 MB/s | 3.6 MB 00:01 2023-12-12T09:46:38.852 INFO:teuthology.orchestra.run.smithi111.stdout:(31/132): ceph-mgr-modules-core-19.0.0-96.gf1fb 1.4 MB/s | 243 kB 00:00 2023-12-12T09:46:38.982 INFO:teuthology.orchestra.run.smithi111.stdout:(32/132): ceph-mgr-rook-19.0.0-96.gf1fbf3c5.el8 389 kB/s | 50 kB 00:00 2023-12-12T09:46:38.982 INFO:teuthology.orchestra.run.smithi173.stdout:(29/132): ceph-mgr-dashboard-19.0.0-96.gf1fbf3c 3.9 MB/s | 1.8 MB 00:00 2023-12-12T09:46:38.994 INFO:teuthology.orchestra.run.smithi042.stdout:(18/132): python3-ceph-argparse-19.0.0-96.gf1fb 132 kB/s | 45 kB 00:00 2023-12-12T09:46:39.070 INFO:teuthology.orchestra.run.smithi111.stdout:(33/132): ceph-prometheus-alerts-19.0.0-96.gf1f 171 kB/s | 15 kB 00:00 2023-12-12T09:46:39.115 INFO:teuthology.orchestra.run.smithi042.stdout:(19/132): python3-ceph-common-19.0.0-96.gf1fbf3 1.0 MB/s | 128 kB 00:00 2023-12-12T09:46:39.232 INFO:teuthology.orchestra.run.smithi042.stdout:(20/132): python3-cephfs-19.0.0-96.gf1fbf3c5.el 1.6 MB/s | 197 kB 00:00 2023-12-12T09:46:39.262 INFO:teuthology.orchestra.run.smithi111.stdout:(34/132): ceph-volume-19.0.0-96.gf1fbf3c5.el8.n 1.3 MB/s | 262 kB 00:00 2023-12-12T09:46:39.349 INFO:teuthology.orchestra.run.smithi042.stdout:(21/132): python3-rados-19.0.0-96.gf1fbf3c5.el8 3.1 MB/s | 371 kB 00:00 2023-12-12T09:46:39.357 INFO:teuthology.orchestra.run.smithi173.stdout:(30/132): rbd-mirror-19.0.0-96.gf1fbf3c5.el8.x8 2.2 MB/s | 3.6 MB 00:01 2023-12-12T09:46:39.457 INFO:teuthology.orchestra.run.smithi042.stdout:(22/132): python3-rbd-19.0.0-96.gf1fbf3c5.el8.x 3.3 MB/s | 364 kB 00:00 2023-12-12T09:46:39.545 INFO:teuthology.orchestra.run.smithi111.stdout:(35/132): cephadm-19.0.0-96.gf1fbf3c5.el8.noarc 1.4 MB/s | 416 kB 00:00 2023-12-12T09:46:39.549 INFO:teuthology.orchestra.run.smithi173.stdout:(31/132): ceph-mgr-modules-core-19.0.0-96.gf1fb 1.2 MB/s | 243 kB 00:00 2023-12-12T09:46:39.557 INFO:teuthology.orchestra.run.smithi042.stdout:(23/132): python3-rgw-19.0.0-96.gf1fbf3c5.el8.x 1.0 MB/s | 106 kB 00:00 2023-12-12T09:46:39.612 INFO:teuthology.orchestra.run.smithi111.stdout:(36/132): cephfs-top-19.0.0-96.gf1fbf3c5.el8.no 273 kB/s | 18 kB 00:00 2023-12-12T09:46:39.633 INFO:teuthology.orchestra.run.smithi173.stdout:(32/132): ceph-mgr-rook-19.0.0-96.gf1fbf3c5.el8 601 kB/s | 50 kB 00:00 2023-12-12T09:46:39.649 INFO:teuthology.orchestra.run.smithi042.stdout:(24/132): rbd-fuse-19.0.0-96.gf1fbf3c5.el8.x86_ 1.0 MB/s | 93 kB 00:00 2023-12-12T09:46:39.706 INFO:teuthology.orchestra.run.smithi111.stdout:(37/132): gperftools-libs-2.7-9.el8.x86_64.rpm 3.3 MB/s | 306 kB 00:00 2023-12-12T09:46:39.733 INFO:teuthology.orchestra.run.smithi173.stdout:(33/132): ceph-prometheus-alerts-19.0.0-96.gf1f 150 kB/s | 15 kB 00:00 2023-12-12T09:46:39.738 INFO:teuthology.orchestra.run.smithi111.stdout:(38/132): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2023-12-12T09:46:39.763 INFO:teuthology.orchestra.run.smithi111.stdout:(39/132): libunwind-1.3.1-3.el8.x86_64.rpm 3.0 MB/s | 75 kB 00:00 2023-12-12T09:46:39.805 INFO:teuthology.orchestra.run.smithi111.stdout:(40/132): luarocks-3.5.0-1.el8.noarch.rpm 3.9 MB/s | 167 kB 00:00 2023-12-12T09:46:39.847 INFO:teuthology.orchestra.run.smithi111.stdout:(41/132): python3-asyncssh-2.7.0-2.el8.noarch.r 11 MB/s | 476 kB 00:00 2023-12-12T09:46:39.873 INFO:teuthology.orchestra.run.smithi111.stdout:(42/132): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.7 MB/s | 44 kB 00:00 2023-12-12T09:46:39.906 INFO:teuthology.orchestra.run.smithi111.stdout:(43/132): python3-beautifulsoup4-4.6.3-2.el8.1. 5.4 MB/s | 185 kB 00:00 2023-12-12T09:46:39.926 INFO:teuthology.orchestra.run.smithi173.stdout:(34/132): ceph-volume-19.0.0-96.gf1fbf3c5.el8.n 1.3 MB/s | 262 kB 00:00 2023-12-12T09:46:39.957 INFO:teuthology.orchestra.run.smithi111.stdout:(44/132): python3-cachetools-3.1.1-4.el8.noarch 660 kB/s | 33 kB 00:00 2023-12-12T09:46:39.982 INFO:teuthology.orchestra.run.smithi111.stdout:(45/132): python3-certifi-2018.10.15-7.el8.noar 657 kB/s | 16 kB 00:00 2023-12-12T09:46:40.066 INFO:teuthology.orchestra.run.smithi111.stdout:(46/132): ceph-mgr-diskprediction-local-19.0.0- 3.9 MB/s | 7.4 MB 00:01 2023-12-12T09:46:40.091 INFO:teuthology.orchestra.run.smithi111.stdout:(47/132): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-12-12T09:46:40.124 INFO:teuthology.orchestra.run.smithi111.stdout:(48/132): python3-google-auth-1.1.1-10.el8.noar 2.4 MB/s | 81 kB 00:00 2023-12-12T09:46:40.133 INFO:teuthology.orchestra.run.smithi173.stdout:(35/132): cephadm-19.0.0-96.gf1fbf3c5.el8.noarc 2.0 MB/s | 416 kB 00:00 2023-12-12T09:46:40.166 INFO:teuthology.orchestra.run.smithi111.stdout:(49/132): python3-jaraco-6.2-6.el8.noarch.rpm 268 kB/s | 11 kB 00:00 2023-12-12T09:46:40.192 INFO:teuthology.orchestra.run.smithi111.stdout:(50/132): python3-cherrypy-18.4.0-1.el8.noarch. 3.0 MB/s | 384 kB 00:00 2023-12-12T09:46:40.217 INFO:teuthology.orchestra.run.smithi111.stdout:(51/132): python3-jaraco-functools-2.0-4.el8.no 350 kB/s | 18 kB 00:00 2023-12-12T09:46:40.234 INFO:teuthology.orchestra.run.smithi173.stdout:(36/132): cephfs-top-19.0.0-96.gf1fbf3c5.el8.no 181 kB/s | 18 kB 00:00 2023-12-12T09:46:40.259 INFO:teuthology.orchestra.run.smithi111.stdout:(52/132): python3-kubernetes-11.0.0-6.el8.noarc 17 MB/s | 1.2 MB 00:00 2023-12-12T09:46:40.284 INFO:teuthology.orchestra.run.smithi111.stdout:(53/132): python3-logutils-0.3.5-11.el8.noarch. 727 kB/s | 49 kB 00:00 2023-12-12T09:46:40.309 INFO:teuthology.orchestra.run.smithi111.stdout:(54/132): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-12-12T09:46:40.316 INFO:teuthology.orchestra.run.smithi042.stdout:(25/132): rbd-mirror-19.0.0-96.gf1fbf3c5.el8.x8 5.4 MB/s | 3.6 MB 00:00 2023-12-12T09:46:40.335 INFO:teuthology.orchestra.run.smithi111.stdout:(55/132): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2023-12-12T09:46:40.342 INFO:teuthology.orchestra.run.smithi173.stdout:(37/132): gperftools-libs-2.7-9.el8.x86_64.rpm 2.8 MB/s | 306 kB 00:00 2023-12-12T09:46:40.360 INFO:teuthology.orchestra.run.smithi111.stdout:(56/132): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-12-12T09:46:40.385 INFO:teuthology.orchestra.run.smithi111.stdout:(57/132): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-12-12T09:46:40.408 INFO:teuthology.orchestra.run.smithi042.stdout:(26/132): rbd-nbd-19.0.0-96.gf1fbf3c5.el8.x86_6 1.9 MB/s | 181 kB 00:00 2023-12-12T09:46:40.434 INFO:teuthology.orchestra.run.smithi173.stdout:(38/132): liboath-2.6.2-3.el8.x86_64.rpm 642 kB/s | 59 kB 00:00 2023-12-12T09:46:40.436 INFO:teuthology.orchestra.run.smithi111.stdout:(58/132): python3-repoze-lru-0.7-6.el8.noarch.r 448 kB/s | 34 kB 00:00 2023-12-12T09:46:40.461 INFO:teuthology.orchestra.run.smithi111.stdout:(59/132): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-12-12T09:46:40.478 INFO:teuthology.orchestra.run.smithi111.stdout:(60/132): python3-rsa-4.9-2.el8.noarch.rpm 1.5 MB/s | 64 kB 00:00 2023-12-12T09:46:40.492 INFO:teuthology.orchestra.run.smithi042.stdout:(27/132): ceph-grafana-dashboards-19.0.0-96.gf1 337 kB/s | 28 kB 00:00 2023-12-12T09:46:40.503 INFO:teuthology.orchestra.run.smithi111.stdout:(61/132): python3-simplegeneric-0.8.1-17.el8.no 449 kB/s | 19 kB 00:00 2023-12-12T09:46:40.526 INFO:teuthology.orchestra.run.smithi173.stdout:(39/132): libunwind-1.3.1-3.el8.x86_64.rpm 823 kB/s | 75 kB 00:00 2023-12-12T09:46:40.528 INFO:teuthology.orchestra.run.smithi111.stdout:(62/132): python3-singledispatch-3.4.0.3-18.el8 478 kB/s | 24 kB 00:00 2023-12-12T09:46:40.554 INFO:teuthology.orchestra.run.smithi111.stdout:(63/132): python3-tempora-1.14.1-5.el8.noarch.r 581 kB/s | 29 kB 00:00 2023-12-12T09:46:40.579 INFO:teuthology.orchestra.run.smithi111.stdout:(64/132): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2023-12-12T09:46:40.598 INFO:teuthology.orchestra.run.smithi042.stdout:(28/132): ceph-mgr-cephadm-19.0.0-96.gf1fbf3c5. 1.2 MB/s | 133 kB 00:00 2023-12-12T09:46:40.604 INFO:teuthology.orchestra.run.smithi111.stdout:(65/132): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-12-12T09:46:40.630 INFO:teuthology.orchestra.run.smithi111.stdout:(66/132): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-12-12T09:46:40.651 INFO:teuthology.orchestra.run.smithi173.stdout:(40/132): luarocks-3.5.0-1.el8.noarch.rpm 1.3 MB/s | 167 kB 00:00 2023-12-12T09:46:40.655 INFO:teuthology.orchestra.run.smithi111.stdout:(67/132): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-12-12T09:46:40.706 INFO:teuthology.orchestra.run.smithi111.stdout:(68/132): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2023-12-12T09:46:40.731 INFO:teuthology.orchestra.run.smithi111.stdout:(69/132): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-12-12T09:46:40.756 INFO:teuthology.orchestra.run.smithi111.stdout:(70/132): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-12-12T09:46:40.798 INFO:teuthology.orchestra.run.smithi111.stdout:(71/132): thrift-0.13.0-2.el8.x86_64.rpm 19 MB/s | 1.7 MB 00:00 2023-12-12T09:46:40.802 INFO:teuthology.orchestra.run.smithi173.stdout:(41/132): python3-asyncssh-2.7.0-2.el8.noarch.r 3.1 MB/s | 476 kB 00:00 2023-12-12T09:46:40.848 INFO:teuthology.orchestra.run.smithi111.stdout:(72/132): librabbitmq-0.9.0-3.el8.x86_64.rpm 510 kB/s | 47 kB 00:00 2023-12-12T09:46:40.902 INFO:teuthology.orchestra.run.smithi173.stdout:(42/132): ceph-mgr-diskprediction-local-19.0.0- 3.8 MB/s | 7.4 MB 00:01 2023-12-12T09:46:40.905 INFO:teuthology.orchestra.run.smithi111.stdout:(73/132): libselinux-devel-2.9-5.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2023-12-12T09:46:40.934 INFO:teuthology.orchestra.run.smithi111.stdout:(74/132): python3-jwt-1.6.1-2.el8.noarch.rpm 510 kB/s | 43 kB 00:00 2023-12-12T09:46:40.951 INFO:teuthology.orchestra.run.smithi042.stdout:(29/132): ceph-mgr-dashboard-19.0.0-96.gf1fbf3c 5.0 MB/s | 1.8 MB 00:00 2023-12-12T09:46:40.958 INFO:teuthology.orchestra.run.smithi111.stdout:(75/132): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.1 MB/s | 220 kB 00:00 2023-12-12T09:46:40.977 INFO:teuthology.orchestra.run.smithi173.stdout:(43/132): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 249 kB/s | 44 kB 00:00 2023-12-12T09:46:40.983 INFO:teuthology.orchestra.run.smithi111.stdout:(76/132): keyutils-libs-devel-1.5.10-9.el8.x86_ 961 kB/s | 48 kB 00:00 2023-12-12T09:46:41.003 INFO:teuthology.orchestra.run.smithi173.stdout:(44/132): python3-beautifulsoup4-4.6.3-2.el8.1. 1.9 MB/s | 185 kB 00:00 2023-12-12T09:46:41.008 INFO:teuthology.orchestra.run.smithi111.stdout:(77/132): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-12-12T09:46:41.033 INFO:teuthology.orchestra.run.smithi111.stdout:(78/132): python3-requests-oauthlib-1.0.0-1.el8 849 kB/s | 43 kB 00:00 2023-12-12T09:46:41.059 INFO:teuthology.orchestra.run.smithi111.stdout:(79/132): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-12-12T09:46:41.062 INFO:teuthology.orchestra.run.smithi173.stdout:(45/132): python3-cachetools-3.1.1-4.el8.noarch 395 kB/s | 33 kB 00:00 2023-12-12T09:46:41.079 INFO:teuthology.orchestra.run.smithi173.stdout:(46/132): python3-certifi-2018.10.15-7.el8.noar 231 kB/s | 16 kB 00:00 2023-12-12T09:46:41.084 INFO:teuthology.orchestra.run.smithi111.stdout:(80/132): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-12-12T09:46:41.118 INFO:teuthology.orchestra.run.smithi111.stdout:(81/132): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-12-12T09:46:41.135 INFO:teuthology.orchestra.run.smithi042.stdout:(30/132): ceph-radosgw-19.0.0-96.gf1fbf3c5.el8. 2.3 MB/s | 14 MB 00:06 2023-12-12T09:46:41.135 INFO:teuthology.orchestra.run.smithi111.stdout:(82/132): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-12-12T09:46:41.169 INFO:teuthology.orchestra.run.smithi111.stdout:(83/132): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-12-12T09:46:41.187 INFO:teuthology.orchestra.run.smithi173.stdout:(47/132): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-12-12T09:46:41.210 INFO:teuthology.orchestra.run.smithi111.stdout:(84/132): python3-cryptography-3.2.1-5.el8.x86_ 7.3 MB/s | 559 kB 00:00 2023-12-12T09:46:41.221 INFO:teuthology.orchestra.run.smithi173.stdout:(48/132): python3-cherrypy-18.4.0-1.el8.noarch. 2.7 MB/s | 384 kB 00:00 2023-12-12T09:46:41.236 INFO:teuthology.orchestra.run.smithi111.stdout:(85/132): libverto-devel-0.3.0-5.el8.x86_64.rpm 268 kB/s | 18 kB 00:00 2023-12-12T09:46:41.251 INFO:teuthology.orchestra.run.smithi042.stdout:(31/132): ceph-mgr-modules-core-19.0.0-96.gf1fb 2.0 MB/s | 243 kB 00:00 2023-12-12T09:46:41.269 INFO:teuthology.orchestra.run.smithi111.stdout:(86/132): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-12-12T09:46:41.295 INFO:teuthology.orchestra.run.smithi111.stdout:(87/132): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-12-12T09:46:41.296 INFO:teuthology.orchestra.run.smithi173.stdout:(49/132): python3-google-auth-1.1.1-10.el8.noar 750 kB/s | 81 kB 00:00 2023-12-12T09:46:41.311 INFO:teuthology.orchestra.run.smithi042.stdout:(32/132): ceph-mgr-rook-19.0.0-96.gf1fbf3c5.el8 858 kB/s | 50 kB 00:00 2023-12-12T09:46:41.321 INFO:teuthology.orchestra.run.smithi173.stdout:(50/132): python3-jaraco-6.2-6.el8.noarch.rpm 116 kB/s | 11 kB 00:00 2023-12-12T09:46:41.329 INFO:teuthology.orchestra.run.smithi111.stdout:(88/132): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.1 MB/s | 72 kB 00:00 2023-12-12T09:46:41.369 INFO:teuthology.orchestra.run.smithi042.stdout:(33/132): ceph-prometheus-alerts-19.0.0-96.gf1f 258 kB/s | 15 kB 00:00 2023-12-12T09:46:41.396 INFO:teuthology.orchestra.run.smithi111.stdout:(89/132): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-12-12T09:46:41.410 INFO:teuthology.orchestra.run.smithi173.stdout:(51/132): python3-jaraco-functools-2.0-4.el8.no 155 kB/s | 18 kB 00:00 2023-12-12T09:46:41.437 INFO:teuthology.orchestra.run.smithi111.stdout:(90/132): openssl-devel-1.1.1k-7.el8_6.x86_64.r 14 MB/s | 2.3 MB 00:00 2023-12-12T09:46:41.463 INFO:teuthology.orchestra.run.smithi111.stdout:(91/132): elfutils-libelf-devel-0.186-1.el8.x86 896 kB/s | 60 kB 00:00 2023-12-12T09:46:41.469 INFO:teuthology.orchestra.run.smithi042.stdout:(34/132): ceph-volume-19.0.0-96.gf1fbf3c5.el8.n 2.6 MB/s | 262 kB 00:00 2023-12-12T09:46:41.488 INFO:teuthology.orchestra.run.smithi111.stdout:(92/132): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-12-12T09:46:41.505 INFO:teuthology.orchestra.run.smithi173.stdout:(52/132): python3-logutils-0.3.5-11.el8.noarch. 514 kB/s | 49 kB 00:00 2023-12-12T09:46:41.539 INFO:teuthology.orchestra.run.smithi173.stdout:(53/132): python3-kubernetes-11.0.0-6.el8.noarc 5.5 MB/s | 1.2 MB 00:00 2023-12-12T09:46:41.595 INFO:teuthology.orchestra.run.smithi042.stdout:(35/132): cephadm-19.0.0-96.gf1fbf3c5.el8.noarc 3.3 MB/s | 416 kB 00:00 2023-12-12T09:46:41.631 INFO:teuthology.orchestra.run.smithi173.stdout:(54/132): python3-more-itertools-7.2.0-3.el8.no 470 kB/s | 59 kB 00:00 2023-12-12T09:46:41.656 INFO:teuthology.orchestra.run.smithi173.stdout:(55/132): python3-natsort-7.1.1-2.el8.noarch.rp 536 kB/s | 60 kB 00:00 2023-12-12T09:46:41.661 INFO:teuthology.orchestra.run.smithi042.stdout:(36/132): cephfs-top-19.0.0-96.gf1fbf3c5.el8.no 272 kB/s | 18 kB 00:00 2023-12-12T09:46:41.714 INFO:teuthology.orchestra.run.smithi111.stdout:(93/132): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 230 kB/s | 58 kB 00:00 2023-12-12T09:46:41.739 INFO:teuthology.orchestra.run.smithi111.stdout:(94/132): libcom_err-devel-1.45.6-4.el8.x86_64. 154 kB/s | 39 kB 00:00 2023-12-12T09:46:41.740 INFO:teuthology.orchestra.run.smithi173.stdout:(56/132): python3-portend-2.6-1.el8.noarch.rpm 198 kB/s | 16 kB 00:00 2023-12-12T09:46:41.772 INFO:teuthology.orchestra.run.smithi111.stdout:(95/132): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-12-12T09:46:41.798 INFO:teuthology.orchestra.run.smithi111.stdout:(96/132): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-12-12T09:46:41.798 INFO:teuthology.orchestra.run.smithi173.stdout:(57/132): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-12-12T09:46:41.844 INFO:teuthology.orchestra.run.smithi111.stdout:(97/132): NetworkManager-initscripts-updown-1.3 1.9 MB/s | 138 kB 00:00 2023-12-12T09:46:41.857 INFO:teuthology.orchestra.run.smithi173.stdout:(58/132): python3-repoze-lru-0.7-6.el8.noarch.r 288 kB/s | 34 kB 00:00 2023-12-12T09:46:41.890 INFO:teuthology.orchestra.run.smithi111.stdout:(98/132): python3-pyasn1-modules-0.3.7-6.el8.no 2.4 MB/s | 110 kB 00:00 2023-12-12T09:46:41.907 INFO:teuthology.orchestra.run.smithi173.stdout:(59/132): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-12-12T09:46:41.924 INFO:teuthology.orchestra.run.smithi111.stdout:(99/132): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-12-12T09:46:41.966 INFO:teuthology.orchestra.run.smithi111.stdout:(100/132): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-12-12T09:46:41.974 INFO:teuthology.orchestra.run.smithi173.stdout:(60/132): python3-rsa-4.9-2.el8.noarch.rpm 551 kB/s | 64 kB 00:00 2023-12-12T09:46:42.000 INFO:teuthology.orchestra.run.smithi173.stdout:(61/132): python3-simplegeneric-0.8.1-17.el8.no 204 kB/s | 19 kB 00:00 2023-12-12T09:46:42.008 INFO:teuthology.orchestra.run.smithi111.stdout:(101/132): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-12-12T09:46:42.041 INFO:teuthology.orchestra.run.smithi111.stdout:(102/132): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-12-12T09:46:42.075 INFO:teuthology.orchestra.run.smithi173.stdout:(62/132): python3-singledispatch-3.4.0.3-18.el8 239 kB/s | 24 kB 00:00 2023-12-12T09:46:42.100 INFO:teuthology.orchestra.run.smithi111.stdout:(103/132): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-12-12T09:46:42.125 INFO:teuthology.orchestra.run.smithi173.stdout:(63/132): python3-tempora-1.14.1-5.el8.noarch.r 242 kB/s | 29 kB 00:00 2023-12-12T09:46:42.142 INFO:teuthology.orchestra.run.smithi111.stdout:(104/132): python3-jinja2-2.10.1-3.el8.noarch.r 13 MB/s | 538 kB 00:00 2023-12-12T09:46:42.175 INFO:teuthology.orchestra.run.smithi111.stdout:(105/132): python3-mako-1.0.6-13.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-12-12T09:46:42.184 INFO:teuthology.orchestra.run.smithi173.stdout:(64/132): python3-trustme-0.6.0-4.el8.noarch.rp 254 kB/s | 27 kB 00:00 2023-12-12T09:46:42.218 INFO:teuthology.orchestra.run.smithi111.stdout:(106/132): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-12-12T09:46:42.262 INFO:teuthology.orchestra.run.smithi042.stdout:(37/132): ceph-test-19.0.0-96.gf1fbf3c5.el8.x86 8.2 MB/s | 49 MB 00:06 2023-12-12T09:46:42.292 INFO:teuthology.orchestra.run.smithi173.stdout:(65/132): python3-webob-1.8.5-1.el8.1.noarch.rp 2.3 MB/s | 251 kB 00:00 2023-12-12T09:46:42.326 INFO:teuthology.orchestra.run.smithi173.stdout:(66/132): python3-waitress-1.4.3-1.el8.noarch.r 1.3 MB/s | 255 kB 00:00 2023-12-12T09:46:42.362 INFO:teuthology.orchestra.run.smithi042.stdout:(38/132): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2023-12-12T09:46:42.402 INFO:teuthology.orchestra.run.smithi111.stdout:(107/132): python3-scipy-1.0.0-21.module+el8.5. 23 MB/s | 14 MB 00:00 2023-12-12T09:46:42.402 INFO:teuthology.orchestra.run.smithi173.stdout:(67/132): python3-websocket-client-0.56.0-5.el8 564 kB/s | 61 kB 00:00 2023-12-12T09:46:42.426 INFO:teuthology.orchestra.run.smithi173.stdout:(68/132): python3-webtest-2.0.33-1.el8.noarch.r 892 kB/s | 85 kB 00:00 2023-12-12T09:46:42.435 INFO:teuthology.orchestra.run.smithi111.stdout:(108/132): automake-1.16.1-7.el8.noarch.rpm 3.2 MB/s | 713 kB 00:00 2023-12-12T09:46:42.460 INFO:teuthology.orchestra.run.smithi111.stdout:(109/132): python3-jsonpatch-1.21-2.el8.noarch. 1.0 MB/s | 27 kB 00:00 2023-12-12T09:46:42.493 INFO:teuthology.orchestra.run.smithi173.stdout:(69/132): python3-zc-lockfile-2.0-2.el8.noarch. 251 kB/s | 23 kB 00:00 2023-12-12T09:46:42.502 INFO:teuthology.orchestra.run.smithi111.stdout:(110/132): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-12T09:46:42.519 INFO:teuthology.orchestra.run.smithi173.stdout:(70/132): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-12-12T09:46:42.536 INFO:teuthology.orchestra.run.smithi111.stdout:(111/132): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-12-12T09:46:42.561 INFO:teuthology.orchestra.run.smithi173.stdout:(71/132): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-12-12T09:46:42.595 INFO:teuthology.orchestra.run.smithi173.stdout:(72/132): libselinux-devel-2.9-5.el8.x86_64.rpm 5.8 MB/s | 200 kB 00:00 2023-12-12T09:46:42.603 INFO:teuthology.orchestra.run.smithi111.stdout:(112/132): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-12-12T09:46:42.628 INFO:teuthology.orchestra.run.smithi173.stdout:(73/132): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-12-12T09:46:42.645 INFO:teuthology.orchestra.run.smithi111.stdout:(113/132): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2023-12-12T09:46:42.670 INFO:teuthology.orchestra.run.smithi173.stdout:(74/132): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-12-12T09:46:42.687 INFO:teuthology.orchestra.run.smithi111.stdout:(114/132): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-12-12T09:46:42.704 INFO:teuthology.orchestra.run.smithi173.stdout:(75/132): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.4 MB/s | 48 kB 00:00 2023-12-12T09:46:42.738 INFO:teuthology.orchestra.run.smithi173.stdout:(76/132): libsepol-devel-2.9-3.el8.x86_64.rpm 2.6 MB/s | 87 kB 00:00 2023-12-12T09:46:42.745 INFO:teuthology.orchestra.run.smithi111.stdout:(115/132): python3-numpy-1.14.3-10.el8.x86_64.r 18 MB/s | 3.7 MB 00:00 2023-12-12T09:46:42.771 INFO:teuthology.orchestra.run.smithi111.stdout:(116/132): uuid-1.6.2-43.el8.x86_64.rpm 764 kB/s | 64 kB 00:00 2023-12-12T09:46:42.782 INFO:teuthology.orchestra.run.smithi173.stdout:(77/132): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-12-12T09:46:42.821 INFO:teuthology.orchestra.run.smithi111.stdout:(117/132): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.6 MB/s | 126 kB 00:00 2023-12-12T09:46:42.838 INFO:teuthology.orchestra.run.smithi111.stdout:(118/132): python3-jsonpointer-1.10-11.el8.noar 298 kB/s | 20 kB 00:00 2023-12-12T09:46:42.846 INFO:teuthology.orchestra.run.smithi042.stdout:(39/132): liboath-2.6.2-3.el8.x86_64.rpm 101 kB/s | 59 kB 00:00 2023-12-12T09:46:42.863 INFO:teuthology.orchestra.run.smithi111.stdout:(119/132): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-12-12T09:46:42.897 INFO:teuthology.orchestra.run.smithi111.stdout:(120/132): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-12-12T09:46:42.913 INFO:teuthology.orchestra.run.smithi042.stdout:(40/132): gperftools-libs-2.7-9.el8.x86_64.rpm 245 kB/s | 306 kB 00:01 2023-12-12T09:46:42.938 INFO:teuthology.orchestra.run.smithi042.stdout:(41/132): libunwind-1.3.1-3.el8.x86_64.rpm 131 kB/s | 75 kB 00:00 2023-12-12T09:46:42.939 INFO:teuthology.orchestra.run.smithi111.stdout:(121/132): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-12-12T09:46:42.974 INFO:teuthology.orchestra.run.smithi173.stdout:(78/132): python3-pycparser-2.14-14.el8.noarch. 561 kB/s | 109 kB 00:00 2023-12-12T09:46:43.014 INFO:teuthology.orchestra.run.smithi173.stdout:(79/132): thrift-0.13.0-2.el8.x86_64.rpm 3.0 MB/s | 1.7 MB 00:00 2023-12-12T09:46:43.039 INFO:teuthology.orchestra.run.smithi111.stdout:(122/132): openblas-threads-0.3.15-3.el8.x86_64 28 MB/s | 4.9 MB 00:00 2023-12-12T09:46:43.047 INFO:teuthology.orchestra.run.smithi042.stdout:(42/132): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 402 kB/s | 44 kB 00:00 2023-12-12T09:46:43.047 INFO:teuthology.orchestra.run.smithi173.stdout:(80/132): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.1 MB/s | 229 kB 00:00 2023-12-12T09:46:43.072 INFO:teuthology.orchestra.run.smithi111.stdout:(123/132): jq-1.6-3.el8.x86_64.rpm 5.9 MB/s | 202 kB 00:00 2023-12-12T09:46:43.089 INFO:teuthology.orchestra.run.smithi173.stdout:(81/132): libaio-devel-0.3.112-1.el8.x86_64.rpm 516 kB/s | 19 kB 00:00 2023-12-12T09:46:43.123 INFO:teuthology.orchestra.run.smithi173.stdout:(82/132): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.0 MB/s | 560 kB 00:00 2023-12-12T09:46:43.147 INFO:teuthology.orchestra.run.smithi042.stdout:(43/132): luarocks-3.5.0-1.el8.noarch.rpm 555 kB/s | 167 kB 00:00 2023-12-12T09:46:43.148 INFO:teuthology.orchestra.run.smithi173.stdout:(83/132): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2023-12-12T09:46:43.157 INFO:teuthology.orchestra.run.smithi111.stdout:(124/132): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-12-12T09:46:43.190 INFO:teuthology.orchestra.run.smithi111.stdout:(125/132): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-12-12T09:46:43.190 INFO:teuthology.orchestra.run.smithi173.stdout:(84/132): python3-cryptography-3.2.1-5.el8.x86_ 8.2 MB/s | 559 kB 00:00 2023-12-12T09:46:43.215 INFO:teuthology.orchestra.run.smithi173.stdout:(85/132): libverto-devel-0.3.0-5.el8.x86_64.rpm 267 kB/s | 18 kB 00:00 2023-12-12T09:46:43.249 INFO:teuthology.orchestra.run.smithi111.stdout:(126/132): postgresql-contrib-10.21-2.module+el 14 MB/s | 808 kB 00:00 2023-12-12T09:46:43.249 INFO:teuthology.orchestra.run.smithi173.stdout:(86/132): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-12-12T09:46:43.255 INFO:teuthology.orchestra.run.smithi042.stdout:(44/132): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2023-12-12T09:46:43.274 INFO:teuthology.orchestra.run.smithi173.stdout:(87/132): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-12-12T09:46:43.308 INFO:teuthology.orchestra.run.smithi173.stdout:(88/132): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.1 MB/s | 72 kB 00:00 2023-12-12T09:46:43.315 INFO:teuthology.orchestra.run.smithi111.stdout:(127/132): postgresql-10.21-2.module+el8.6.0+15 23 MB/s | 1.5 MB 00:00 2023-12-12T09:46:43.348 INFO:teuthology.orchestra.run.smithi042.stdout:(45/132): python3-beautifulsoup4-4.6.3-2.el8.1. 616 kB/s | 185 kB 00:00 2023-12-12T09:46:43.350 INFO:teuthology.orchestra.run.smithi173.stdout:(89/132): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-12-12T09:46:43.365 INFO:teuthology.orchestra.run.smithi042.stdout:(46/132): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-12-12T09:46:43.383 INFO:teuthology.orchestra.run.smithi111.stdout:(128/132): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-12-12T09:46:43.400 INFO:teuthology.orchestra.run.smithi173.stdout:(90/132): openssl-devel-1.1.1k-7.el8_6.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-12-12T09:46:43.425 INFO:teuthology.orchestra.run.smithi173.stdout:(91/132): elfutils-libelf-devel-0.186-1.el8.x86 796 kB/s | 60 kB 00:00 2023-12-12T09:46:43.451 INFO:teuthology.orchestra.run.smithi173.stdout:(92/132): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-12-12T09:46:43.476 INFO:teuthology.orchestra.run.smithi173.stdout:(93/132): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-12-12T09:46:43.522 INFO:teuthology.orchestra.run.smithi173.stdout:(94/132): libcom_err-devel-1.45.6-4.el8.x86_64. 545 kB/s | 39 kB 00:00 2023-12-12T09:46:43.560 INFO:teuthology.orchestra.run.smithi173.stdout:(95/132): libgfortran-8.5.0-10.1.el8_6.x86_64.r 7.5 MB/s | 644 kB 00:00 2023-12-12T09:46:43.585 INFO:teuthology.orchestra.run.smithi173.stdout:(96/132): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.7 MB/s | 171 kB 00:00 2023-12-12T09:46:43.619 INFO:teuthology.orchestra.run.smithi173.stdout:(97/132): NetworkManager-initscripts-updown-1.3 2.3 MB/s | 138 kB 00:00 2023-12-12T09:46:43.632 INFO:teuthology.orchestra.run.smithi042.stdout:(47/132): python3-cheroot-8.5.2-1.el8.noarch.rp 608 kB/s | 173 kB 00:00 2023-12-12T09:46:43.652 INFO:teuthology.orchestra.run.smithi173.stdout:(98/132): python3-pyasn1-modules-0.3.7-6.el8.no 3.2 MB/s | 110 kB 00:00 2023-12-12T09:46:43.657 INFO:teuthology.orchestra.run.smithi042.stdout:(48/132): python3-asyncssh-2.7.0-2.el8.noarch.r 640 kB/s | 476 kB 00:00 2023-12-12T09:46:43.686 INFO:teuthology.orchestra.run.smithi173.stdout:(99/132): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-12-12T09:46:43.730 INFO:teuthology.orchestra.run.smithi173.stdout:(100/132): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.8 MB/s | 259 kB 00:00 2023-12-12T09:46:43.766 INFO:teuthology.orchestra.run.smithi042.stdout:(49/132): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-12-12T09:46:43.772 INFO:teuthology.orchestra.run.smithi173.stdout:(101/132): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-12-12T09:46:43.806 INFO:teuthology.orchestra.run.smithi173.stdout:(102/132): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-12-12T09:46:43.832 INFO:teuthology.orchestra.run.smithi042.stdout:(50/132): python3-google-auth-1.1.1-10.el8.noar 405 kB/s | 81 kB 00:00 2023-12-12T09:46:43.856 INFO:teuthology.orchestra.run.smithi173.stdout:(103/132): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-12-12T09:46:43.866 INFO:teuthology.orchestra.run.smithi042.stdout:(51/132): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-12-12T09:46:43.898 INFO:teuthology.orchestra.run.smithi173.stdout:(104/132): python3-jinja2-2.10.1-3.el8.noarch.r 13 MB/s | 538 kB 00:00 2023-12-12T09:46:43.940 INFO:teuthology.orchestra.run.smithi173.stdout:(105/132): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-12-12T09:46:43.973 INFO:teuthology.orchestra.run.smithi173.stdout:(106/132): oniguruma-6.8.2-2.el8.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2023-12-12T09:46:43.983 INFO:teuthology.orchestra.run.smithi042.stdout:(52/132): python3-logutils-0.3.5-11.el8.noarch. 417 kB/s | 49 kB 00:00 2023-12-12T09:46:44.008 INFO:teuthology.orchestra.run.smithi111.stdout:(129/132): librados2-19.0.0-96.gf1fbf3c5.el8.x8 5.4 MB/s | 3.8 MB 00:00 2023-12-12T09:46:44.008 INFO:teuthology.orchestra.run.smithi042.stdout:(53/132): python3-cherrypy-18.4.0-1.el8.noarch. 598 kB/s | 384 kB 00:00 2023-12-12T09:46:44.050 INFO:teuthology.orchestra.run.smithi111.stdout:(130/132): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2023-12-12T09:46:44.092 INFO:teuthology.orchestra.run.smithi042.stdout:(54/132): python3-more-itertools-7.2.0-3.el8.no 541 kB/s | 59 kB 00:00 2023-12-12T09:46:44.117 INFO:teuthology.orchestra.run.smithi042.stdout:(55/132): python3-natsort-7.1.1-2.el8.noarch.rp 555 kB/s | 60 kB 00:00 2023-12-12T09:46:44.124 INFO:teuthology.orchestra.run.smithi173.stdout:(107/132): python3-scipy-1.0.0-21.module+el8.5. 25 MB/s | 14 MB 00:00 2023-12-12T09:46:44.157 INFO:teuthology.orchestra.run.smithi173.stdout:(108/132): automake-1.16.1-7.el8.noarch.rpm 3.8 MB/s | 713 kB 00:00 2023-12-12T09:46:44.191 INFO:teuthology.orchestra.run.smithi173.stdout:(109/132): python3-jsonpatch-1.21-2.el8.noarch. 796 kB/s | 27 kB 00:00 2023-12-12T09:46:44.226 INFO:teuthology.orchestra.run.smithi042.stdout:(56/132): python3-portend-2.6-1.el8.noarch.rpm 144 kB/s | 16 kB 00:00 2023-12-12T09:46:44.241 INFO:teuthology.orchestra.run.smithi173.stdout:(110/132): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-12-12T09:46:44.275 INFO:teuthology.orchestra.run.smithi173.stdout:(111/132): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-12-12T09:46:44.335 INFO:teuthology.orchestra.run.smithi042.stdout:(57/132): python3-repoze-lru-0.7-6.el8.noarch.r 310 kB/s | 34 kB 00:00 2023-12-12T09:46:44.350 INFO:teuthology.orchestra.run.smithi173.stdout:(112/132): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-12-12T09:46:44.400 INFO:teuthology.orchestra.run.smithi111.stdout:(131/132): librbd1-19.0.0-96.gf1fbf3c5.el8.x86_ 3.6 MB/s | 3.7 MB 00:01 2023-12-12T09:46:44.400 INFO:teuthology.orchestra.run.smithi173.stdout:(113/132): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-12-12T09:46:44.442 INFO:teuthology.orchestra.run.smithi173.stdout:(114/132): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-12-12T09:46:44.501 INFO:teuthology.orchestra.run.smithi173.stdout:(115/132): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2023-12-12T09:46:44.526 INFO:teuthology.orchestra.run.smithi173.stdout:(116/132): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-12-12T09:46:44.552 INFO:teuthology.orchestra.run.smithi173.stdout:(117/132): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-12-12T09:46:44.560 INFO:teuthology.orchestra.run.smithi042.stdout:(58/132): python3-pecan-1.3.2-9.el8.noarch.rpm 605 kB/s | 283 kB 00:00 2023-12-12T09:46:44.577 INFO:teuthology.orchestra.run.smithi173.stdout:(118/132): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-12-12T09:46:44.602 INFO:teuthology.orchestra.run.smithi173.stdout:(119/132): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-12-12T09:46:44.630 INFO:teuthology.orchestra.run.smithi042.stdout:(59/132): python3-routes-2.4.1-12.el8.noarch.rp 672 kB/s | 196 kB 00:00 2023-12-12T09:46:44.644 INFO:teuthology.orchestra.run.smithi173.stdout:(120/132): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-12-12T09:46:44.669 INFO:teuthology.orchestra.run.smithi042.stdout:(60/132): python3-rsa-4.9-2.el8.noarch.rpm 589 kB/s | 64 kB 00:00 2023-12-12T09:46:44.736 INFO:teuthology.orchestra.run.smithi042.stdout:(61/132): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2023-12-12T09:46:44.778 INFO:teuthology.orchestra.run.smithi042.stdout:(62/132): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2023-12-12T09:46:44.875 INFO:teuthology.orchestra.run.smithi042.stdout:(63/132): python3-tempora-1.14.1-5.el8.noarch.r 211 kB/s | 29 kB 00:00 2023-12-12T09:46:44.904 INFO:teuthology.orchestra.run.smithi042.stdout:(64/132): python3-trustme-0.6.0-4.el8.noarch.rp 220 kB/s | 27 kB 00:00 2023-12-12T09:46:45.111 INFO:teuthology.orchestra.run.smithi173.stdout:(121/132): ceph-test-19.0.0-96.gf1fbf3c5.el8.x8 3.9 MB/s | 49 MB 00:12 2023-12-12T09:46:45.136 INFO:teuthology.orchestra.run.smithi173.stdout:(122/132): libpq-13.5-1.el8.x86_64.rpm 402 kB/s | 198 kB 00:00 2023-12-12T09:46:45.187 INFO:teuthology.orchestra.run.smithi173.stdout:(123/132): jq-1.6-3.el8.x86_64.rpm 4.4 MB/s | 202 kB 00:00 2023-12-12T09:46:45.229 INFO:teuthology.orchestra.run.smithi173.stdout:(124/132): librdkafka-0.11.4-3.el8.x86_64.rpm 8.4 MB/s | 354 kB 00:00 2023-12-12T09:46:45.262 INFO:teuthology.orchestra.run.smithi042.stdout:(65/132): python3-waitress-1.4.3-1.el8.noarch.r 660 kB/s | 255 kB 00:00 2023-12-12T09:46:45.309 INFO:teuthology.orchestra.run.smithi111.stdout:(132/132): ceph-test-19.0.0-96.gf1fbf3c5.el8.x8 4.3 MB/s | 49 MB 00:11 2023-12-12T09:46:45.311 INFO:teuthology.orchestra.run.smithi042.stdout:(66/132): python3-webob-1.8.5-1.el8.1.noarch.rp 616 kB/s | 251 kB 00:00 2023-12-12T09:46:45.312 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:46:45.312 INFO:teuthology.orchestra.run.smithi111.stdout:Total 12 MB/s | 209 MB 00:17 2023-12-12T09:46:45.329 INFO:teuthology.orchestra.run.smithi173.stdout:(125/132): openblas-threads-0.3.15-3.el8.x86_64 6.7 MB/s | 4.9 MB 00:00 2023-12-12T09:46:45.388 INFO:teuthology.orchestra.run.smithi042.stdout:(67/132): python3-kubernetes-11.0.0-6.el8.noarc 767 kB/s | 1.2 MB 00:01 2023-12-12T09:46:45.412 INFO:teuthology.orchestra.run.smithi173.stdout:(126/132): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-12-12T09:46:45.413 INFO:teuthology.orchestra.run.smithi042.stdout:(68/132): python3-websocket-client-0.56.0-5.el8 407 kB/s | 61 kB 00:00 2023-12-12T09:46:45.446 INFO:teuthology.orchestra.run.smithi173.stdout:(127/132): postgresql-contrib-10.21-2.module+el 6.8 MB/s | 808 kB 00:00 2023-12-12T09:46:45.505 INFO:teuthology.orchestra.run.smithi173.stdout:(128/132): postgresql-10.21-2.module+el8.6.0+15 16 MB/s | 1.5 MB 00:00 2023-12-12T09:46:45.505 INFO:teuthology.orchestra.run.smithi042.stdout:(69/132): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2023-12-12T09:46:45.531 INFO:teuthology.orchestra.run.smithi042.stdout:(70/132): python3-webtest-2.0.33-1.el8.noarch.r 390 kB/s | 85 kB 00:00 2023-12-12T09:46:45.556 INFO:teuthology.orchestra.run.smithi042.stdout:(71/132): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-12-12T09:46:45.588 INFO:teuthology.orchestra.run.smithi173.stdout:(129/132): postgresql-server-10.21-2.module+el8 14 MB/s | 5.1 MB 00:00 2023-12-12T09:46:45.623 INFO:teuthology.orchestra.run.smithi042.stdout:(72/132): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-12-12T09:46:45.639 INFO:teuthology.orchestra.run.smithi173.stdout:(130/132): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-12-12T09:46:45.657 INFO:teuthology.orchestra.run.smithi042.stdout:(73/132): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-12-12T09:46:45.698 INFO:teuthology.orchestra.run.smithi042.stdout:(74/132): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-12-12T09:46:45.774 INFO:teuthology.orchestra.run.smithi042.stdout:(75/132): socat-1.7.4.1-1.el8.x86_64.rpm 4.2 MB/s | 323 kB 00:00 2023-12-12T09:46:45.807 INFO:teuthology.orchestra.run.smithi042.stdout:(76/132): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-12-12T09:46:45.866 INFO:teuthology.orchestra.run.smithi042.stdout:(77/132): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-12-12T09:46:45.908 INFO:teuthology.orchestra.run.smithi042.stdout:(78/132): python3-jinja2-2.10.1-3.el8.noarch.rp 13 MB/s | 538 kB 00:00 2023-12-12T09:46:45.950 INFO:teuthology.orchestra.run.smithi042.stdout:(79/132): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-12-12T09:46:45.992 INFO:teuthology.orchestra.run.smithi042.stdout:(80/132): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-12-12T09:46:46.042 INFO:teuthology.orchestra.run.smithi042.stdout:(81/132): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-12-12T09:46:46.138 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2023-12-12T09:46:46.172 INFO:teuthology.orchestra.run.smithi173.stdout:(131/132): librados2-19.0.0-96.gf1fbf3c5.el8.x8 5.2 MB/s | 3.8 MB 00:00 2023-12-12T09:46:46.226 INFO:teuthology.orchestra.run.smithi042.stdout:(82/132): python3-scipy-1.0.0-21.module+el8.5.0 20 MB/s | 14 MB 00:00 2023-12-12T09:46:46.251 INFO:teuthology.orchestra.run.smithi042.stdout:(83/132): python3-jsonpatch-1.21-2.el8.noarch.r 1.1 MB/s | 27 kB 00:00 2023-12-12T09:46:46.259 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2023-12-12T09:46:46.259 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2023-12-12T09:46:46.293 INFO:teuthology.orchestra.run.smithi042.stdout:(84/132): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-12-12T09:46:46.319 INFO:teuthology.orchestra.run.smithi042.stdout:(85/132): python3-prettytable-0.7.2-14.el8.noar 1.7 MB/s | 44 kB 00:00 2023-12-12T09:46:46.402 INFO:teuthology.orchestra.run.smithi042.stdout:(86/132): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2023-12-12T09:46:46.444 INFO:teuthology.orchestra.run.smithi042.stdout:(87/132): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-12-12T09:46:46.462 INFO:teuthology.orchestra.run.smithi173.stdout:(132/132): librbd1-19.0.0-96.gf1fbf3c5.el8.x86_ 3.9 MB/s | 3.7 MB 00:00 2023-12-12T09:46:46.465 INFO:teuthology.orchestra.run.smithi173.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:46:46.465 INFO:teuthology.orchestra.run.smithi173.stdout:Total 11 MB/s | 209 MB 00:18 2023-12-12T09:46:46.486 INFO:teuthology.orchestra.run.smithi042.stdout:(88/132): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-12-12T09:46:46.545 INFO:teuthology.orchestra.run.smithi042.stdout:(89/132): python3-numpy-1.14.3-10.el8.x86_64.rp 17 MB/s | 3.7 MB 00:00 2023-12-12T09:46:46.570 INFO:teuthology.orchestra.run.smithi042.stdout:(90/132): uuid-1.6.2-43.el8.x86_64.rpm 764 kB/s | 64 kB 00:00 2023-12-12T09:46:46.596 INFO:teuthology.orchestra.run.smithi042.stdout:(91/132): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.4 MB/s | 126 kB 00:00 2023-12-12T09:46:46.655 INFO:teuthology.orchestra.run.smithi042.stdout:(92/132): python3-jsonpointer-1.10-11.el8.noarc 236 kB/s | 20 kB 00:00 2023-12-12T09:46:46.688 INFO:teuthology.orchestra.run.smithi042.stdout:(93/132): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.1 MB/s | 103 kB 00:00 2023-12-12T09:46:46.722 INFO:teuthology.orchestra.run.smithi042.stdout:(94/132): libtool-2.4.6-25.el8.x86_64.rpm 11 MB/s | 709 kB 00:00 2023-12-12T09:46:46.755 INFO:teuthology.orchestra.run.smithi042.stdout:(95/132): libpq-13.5-1.el8.x86_64.rpm 5.8 MB/s | 198 kB 00:00 2023-12-12T09:46:46.872 INFO:teuthology.orchestra.run.smithi042.stdout:(96/132): openblas-threads-0.3.15-3.el8.x86_64. 26 MB/s | 4.9 MB 00:00 2023-12-12T09:46:46.914 INFO:teuthology.orchestra.run.smithi042.stdout:(97/132): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-12-12T09:46:46.989 INFO:teuthology.orchestra.run.smithi042.stdout:(98/132): openblas-0.3.15-3.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2023-12-12T09:46:47.023 INFO:teuthology.orchestra.run.smithi042.stdout:(99/132): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-12-12T09:46:47.107 INFO:teuthology.orchestra.run.smithi042.stdout:(100/132): postgresql-contrib-10.21-2.module+el 9.5 MB/s | 808 kB 00:00 2023-12-12T09:46:47.174 INFO:teuthology.orchestra.run.smithi042.stdout:(101/132): postgresql-10.21-2.module+el8.6.0+15 23 MB/s | 1.5 MB 00:00 2023-12-12T09:46:47.240 INFO:teuthology.orchestra.run.smithi042.stdout:(102/132): postgresql-server-10.21-2.module+el8 20 MB/s | 5.1 MB 00:00 2023-12-12T09:46:47.266 INFO:teuthology.orchestra.run.smithi042.stdout:(103/132): librabbitmq-0.9.0-3.el8.x86_64.rpm 510 kB/s | 47 kB 00:00 2023-12-12T09:46:47.287 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction check 2023-12-12T09:46:47.291 INFO:teuthology.orchestra.run.smithi042.stdout:(104/132): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-12-12T09:46:47.316 INFO:teuthology.orchestra.run.smithi042.stdout:(105/132): python3-jwt-1.6.1-2.el8.noarch.rpm 846 kB/s | 43 kB 00:00 2023-12-12T09:46:47.342 INFO:teuthology.orchestra.run.smithi042.stdout:(106/132): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-12-12T09:46:47.367 INFO:teuthology.orchestra.run.smithi042.stdout:(107/132): keyutils-libs-devel-1.5.10-9.el8.x86 953 kB/s | 48 kB 00:00 2023-12-12T09:46:47.392 INFO:teuthology.orchestra.run.smithi042.stdout:(108/132): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-12-12T09:46:47.396 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2023-12-12T09:46:47.400 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2023-12-12T09:46:47.409 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction check succeeded. 2023-12-12T09:46:47.410 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction test 2023-12-12T09:46:47.417 INFO:teuthology.orchestra.run.smithi042.stdout:(109/132): python3-requests-oauthlib-1.0.0-1.el 850 kB/s | 43 kB 00:00 2023-12-12T09:46:47.443 INFO:teuthology.orchestra.run.smithi042.stdout:(110/132): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-12-12T09:46:47.468 INFO:teuthology.orchestra.run.smithi042.stdout:(111/132): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-12-12T09:46:47.511 INFO:teuthology.orchestra.run.smithi042.stdout:(112/132): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.1 MB/s | 560 kB 00:00 2023-12-12T09:46:47.561 INFO:teuthology.orchestra.run.smithi042.stdout:(113/132): libaio-devel-0.3.112-1.el8.x86_64.rp 203 kB/s | 19 kB 00:00 2023-12-12T09:46:47.586 INFO:teuthology.orchestra.run.smithi042.stdout:(114/132): python3-oauthlib-2.1.0-1.el8.noarch. 2.0 MB/s | 155 kB 00:00 2023-12-12T09:46:47.620 INFO:teuthology.orchestra.run.smithi042.stdout:(115/132): python3-cryptography-3.2.1-5.el8.x86 9.4 MB/s | 559 kB 00:00 2023-12-12T09:46:47.645 INFO:teuthology.orchestra.run.smithi042.stdout:(116/132): libverto-devel-0.3.0-5.el8.x86_64.rp 305 kB/s | 18 kB 00:00 2023-12-12T09:46:47.687 INFO:teuthology.orchestra.run.smithi042.stdout:(117/132): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2023-12-12T09:46:47.712 INFO:teuthology.orchestra.run.smithi042.stdout:(118/132): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.5 MB/s | 238 kB 00:00 2023-12-12T09:46:47.754 INFO:teuthology.orchestra.run.smithi042.stdout:(119/132): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-12-12T09:46:47.804 INFO:teuthology.orchestra.run.smithi042.stdout:(120/132): openssl-devel-1.1.1k-7.el8_6.x86_64. 20 MB/s | 2.3 MB 00:00 2023-12-12T09:46:47.830 INFO:teuthology.orchestra.run.smithi042.stdout:(121/132): fuse-2.9.7-15.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-12-12T09:46:47.847 INFO:teuthology.orchestra.run.smithi042.stdout:(122/132): elfutils-libelf-devel-0.186-1.el8.x8 1.4 MB/s | 60 kB 00:00 2023-12-12T09:46:47.880 INFO:teuthology.orchestra.run.smithi042.stdout:(123/132): thrift-0.13.0-2.el8.x86_64.rpm 720 kB/s | 1.7 MB 00:02 2023-12-12T09:46:47.898 INFO:teuthology.orchestra.run.smithi042.stdout:(124/132): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-12-12T09:46:47.931 INFO:teuthology.orchestra.run.smithi042.stdout:(125/132): libuuid-devel-2.32.1-35.el8.x86_64.r 974 kB/s | 98 kB 00:00 2023-12-12T09:46:47.948 INFO:teuthology.orchestra.run.smithi042.stdout:(126/132): libcom_err-devel-1.45.6-4.el8.x86_64 574 kB/s | 39 kB 00:00 2023-12-12T09:46:47.998 INFO:teuthology.orchestra.run.smithi042.stdout:(127/132): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-12-12T09:46:48.023 INFO:teuthology.orchestra.run.smithi042.stdout:(128/132): NetworkManager-initscripts-updown-1. 1.8 MB/s | 138 kB 00:00 2023-12-12T09:46:48.057 INFO:teuthology.orchestra.run.smithi042.stdout:(129/132): libgfortran-8.5.0-10.1.el8_6.x86_64. 4.2 MB/s | 644 kB 00:00 2023-12-12T09:46:48.108 INFO:teuthology.orchestra.run.smithi042.stdout:(130/132): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-12-12T09:46:48.358 INFO:teuthology.orchestra.run.smithi042.stdout:(131/132): librados2-19.0.0-96.gf1fbf3c5.el8.x8 11 MB/s | 3.8 MB 00:00 2023-12-12T09:46:48.551 INFO:teuthology.orchestra.run.smithi173.stdout:Transaction test succeeded. 2023-12-12T09:46:48.555 INFO:teuthology.orchestra.run.smithi173.stdout:Running transaction 2023-12-12T09:46:48.583 INFO:teuthology.orchestra.run.smithi042.stdout:(132/132): librbd1-19.0.0-96.gf1fbf3c5.el8.x86_ 6.6 MB/s | 3.7 MB 00:00 2023-12-12T09:46:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:46:48.587 INFO:teuthology.orchestra.run.smithi042.stdout:Total 11 MB/s | 209 MB 00:19 2023-12-12T09:46:49.465 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2023-12-12T09:46:49.603 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2023-12-12T09:46:49.603 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2023-12-12T09:46:50.806 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2023-12-12T09:46:50.810 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2023-12-12T09:46:52.359 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2023-12-12T09:46:52.540 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/135 2023-12-12T09:46:52.732 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/135 2023-12-12T09:46:52.959 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-12T09:46:53.066 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-12T09:46:53.276 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-12T09:46:53.432 INFO:teuthology.orchestra.run.smithi173.stdout: Preparing : 1/1 2023-12-12T09:46:53.712 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-12T09:46:53.934 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 5/135 2023-12-12T09:46:54.096 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/135 2023-12-12T09:46:54.322 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/135 2023-12-12T09:46:54.350 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 5/135 2023-12-12T09:46:54.514 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-12T09:46:54.525 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 6/135 2023-12-12T09:46:54.625 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/135 2023-12-12T09:46:54.655 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 6/135 2023-12-12T09:46:54.847 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-12T09:46:54.859 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 7/135 2023-12-12T09:46:54.938 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 7/135 2023-12-12T09:46:55.097 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:19.0.0-96.gf1fbf3c5.el8.x86_64 8/135 2023-12-12T09:46:55.287 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/135 2023-12-12T09:46:55.505 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 5/135 2023-12-12T09:46:55.852 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2023-12-12T09:46:55.882 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/135 2023-12-12T09:46:55.906 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 5/135 2023-12-12T09:46:56.034 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 10/135 2023-12-12T09:46:56.050 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/135 2023-12-12T09:46:56.072 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 6/135 2023-12-12T09:46:56.109 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 10/135 2023-12-12T09:46:56.185 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 6/135 2023-12-12T09:46:56.213 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/135 2023-12-12T09:46:56.355 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpq-13.5-1.el8.x86_64 11/135 2023-12-12T09:46:56.388 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 7/135 2023-12-12T09:46:56.407 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/135 2023-12-12T09:46:56.468 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 7/135 2023-12-12T09:46:56.552 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/135 2023-12-12T09:46:56.570 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/135 2023-12-12T09:46:56.644 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rados-2:19.0.0-96.gf1fbf3c5.el8.x86_64 8/135 2023-12-12T09:46:56.671 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/135 2023-12-12T09:46:56.919 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/135 2023-12-12T09:46:56.989 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/135 2023-12-12T09:46:57.265 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/135 2023-12-12T09:46:57.353 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/135 2023-12-12T09:46:57.411 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/135 2023-12-12T09:46:57.544 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 6/135 2023-12-12T09:46:57.589 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 10/135 2023-12-12T09:46:57.679 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 10/135 2023-12-12T09:46:57.758 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 15/135 2023-12-12T09:46:57.850 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 15/135 2023-12-12T09:46:57.964 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 6/135 2023-12-12T09:46:57.967 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libpq-13.5-1.el8.x86_64 11/135 2023-12-12T09:46:58.000 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_6 16/135 2023-12-12T09:46:58.154 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_6 16/135 2023-12-12T09:46:58.175 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/135 2023-12-12T09:46:58.210 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 7/135 2023-12-12T09:46:58.359 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 7/135 2023-12-12T09:46:58.380 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/135 2023-12-12T09:46:58.535 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 8/135 2023-12-12T09:46:58.539 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/135 2023-12-12T09:46:58.585 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/135 2023-12-12T09:46:58.633 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 8/135 2023-12-12T09:46:58.771 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/135 2023-12-12T09:46:58.798 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 19/135 2023-12-12T09:46:58.930 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 15/135 2023-12-12T09:46:58.950 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-12T09:46:59.020 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 15/135 2023-12-12T09:46:59.081 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-12T09:46:59.241 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-12T09:46:59.263 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_6 16/135 2023-12-12T09:46:59.394 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_6 16/135 2023-12-12T09:46:59.435 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:19.0.0-96.gf1fbf3c5.el8.x86_64 9/135 2023-12-12T09:46:59.610 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 10/135 2023-12-12T09:46:59.625 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/135 2023-12-12T09:46:59.699 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 10/135 2023-12-12T09:46:59.800 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/135 2023-12-12T09:46:59.853 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-12T09:46:59.926 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpq-13.5-1.el8.x86_64 11/135 2023-12-12T09:47:00.010 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 19/135 2023-12-12T09:47:00.050 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-12T09:47:00.147 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/135 2023-12-12T09:47:00.162 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-12T09:47:00.302 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 20/135 2023-12-12T09:47:00.463 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-12T09:47:00.533 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/135 2023-12-12T09:47:00.644 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-12T09:47:00.756 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/135 2023-12-12T09:47:00.816 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.15-3.el8.x86_64 23/135 2023-12-12T09:47:00.944 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 15/135 2023-12-12T09:47:01.053 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 15/135 2023-12-12T09:47:01.102 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 21/135 2023-12-12T09:47:01.218 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_6 16/135 2023-12-12T09:47:01.340 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_6 16/135 2023-12-12T09:47:01.346 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-12T09:47:01.476 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 17/135 2023-12-12T09:47:01.508 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 23/135 2023-12-12T09:47:01.629 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 17/135 2023-12-12T09:47:01.756 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/135 2023-12-12T09:47:01.826 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 18/135 2023-12-12T09:47:01.940 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 22/135 2023-12-12T09:47:01.963 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/135 2023-12-12T09:47:02.121 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openblas-0.3.15-3.el8.x86_64 23/135 2023-12-12T09:47:02.201 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/135 2023-12-12T09:47:02.397 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x8 27/135 2023-12-12T09:47:02.468 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 18/135 2023-12-12T09:47:02.519 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el8. 28/135 2023-12-12T09:47:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 19/135 2023-12-12T09:47:02.745 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el8.x86_64 29/135 2023-12-12T09:47:02.823 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 23/135 2023-12-12T09:47:02.966 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-12T09:47:03.077 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/135 2023-12-12T09:47:03.278 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 25/135 2023-12-12T09:47:03.291 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 19/135 2023-12-12T09:47:03.460 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.15-3.el8.x86_64 20/135 2023-12-12T09:47:03.565 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/135 2023-12-12T09:47:03.727 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x8 27/135 2023-12-12T09:47:03.865 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el8. 28/135 2023-12-12T09:47:04.093 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el8.x86_64 29/135 2023-12-12T09:47:04.249 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 20/135 2023-12-12T09:47:04.303 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-12T09:47:04.517 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 21/135 2023-12-12T09:47:04.724 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-12T09:47:04.728 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 22/135 2023-12-12T09:47:04.948 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/135 2023-12-12T09:47:05.101 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/135 2023-12-12T09:47:05.468 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 25/135 2023-12-12T09:47:05.697 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/135 2023-12-12T09:47:05.735 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/135 2023-12-12T09:47:06.042 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x8 27/135 2023-12-12T09:47:06.047 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 32/135 2023-12-12T09:47:06.059 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/135 2023-12-12T09:47:06.315 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el8. 28/135 2023-12-12T09:47:06.326 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 33/135 2023-12-12T09:47:06.568 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/135 2023-12-12T09:47:06.810 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/135 2023-12-12T09:47:06.993 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/135 2023-12-12T09:47:07.090 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/135 2023-12-12T09:47:07.234 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 37/135 2023-12-12T09:47:07.242 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el8.x86_64 29/135 2023-12-12T09:47:07.410 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 32/135 2023-12-12T09:47:07.421 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 38/135 2023-12-12T09:47:07.541 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 30/135 2023-12-12T09:47:07.613 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 39/135 2023-12-12T09:47:07.706 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 33/135 2023-12-12T09:47:07.772 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 40/135 2023-12-12T09:47:07.791 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 31/135 2023-12-12T09:47:07.927 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/135 2023-12-12T09:47:07.933 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 41/135 2023-12-12T09:47:08.065 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/135 2023-12-12T09:47:08.131 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/135 2023-12-12T09:47:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-43.el8.x86_64 42/135 2023-12-12T09:47:08.207 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 42/135 2023-12-12T09:47:08.259 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/135 2023-12-12T09:47:08.357 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/135 2023-12-12T09:47:08.428 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 43/135 2023-12-12T09:47:08.589 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 37/135 2023-12-12T09:47:08.660 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.1-7.el8.noarch 44/135 2023-12-12T09:47:08.785 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-rbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 38/135 2023-12-12T09:47:08.815 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-12T09:47:08.893 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-12T09:47:08.960 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : librados-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 39/135 2023-12-12T09:47:09.119 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : jq-1.6-3.el8.x86_64 46/135 2023-12-12T09:47:09.152 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 40/135 2023-12-12T09:47:09.285 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/135 2023-12-12T09:47:09.313 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 41/135 2023-12-12T09:47:09.463 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/135 2023-12-12T09:47:09.520 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : uuid-1.6.2-43.el8.x86_64 42/135 2023-12-12T09:47:09.593 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 42/135 2023-12-12T09:47:09.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 49/135 2023-12-12T09:47:09.850 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.7-15.el8.x86_64 50/135 2023-12-12T09:47:09.865 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 43/135 2023-12-12T09:47:09.992 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 51/135 2023-12-12T09:47:10.082 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : automake-1.16.1-7.el8.noarch 44/135 2023-12-12T09:47:10.142 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/135 2023-12-12T09:47:10.185 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 52/135 2023-12-12T09:47:10.253 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-12T09:47:10.338 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/135 2023-12-12T09:47:10.383 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 53/135 2023-12-12T09:47:10.437 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/135 2023-12-12T09:47:10.601 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : jq-1.6-3.el8.x86_64 46/135 2023-12-12T09:47:10.668 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el8.noarch 35/135 2023-12-12T09:47:10.782 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/135 2023-12-12T09:47:10.797 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 54/135 2023-12-12T09:47:10.821 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/135 2023-12-12T09:47:10.944 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/135 2023-12-12T09:47:11.013 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 37/135 2023-12-12T09:47:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 55/135 2023-12-12T09:47:11.172 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 49/135 2023-12-12T09:47:11.224 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 38/135 2023-12-12T09:47:11.334 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 56/135 2023-12-12T09:47:11.364 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : fuse-2.9.7-15.el8.x86_64 50/135 2023-12-12T09:47:11.371 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 39/135 2023-12-12T09:47:11.515 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 51/135 2023-12-12T09:47:11.585 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/135 2023-12-12T09:47:11.593 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 40/135 2023-12-12T09:47:11.719 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 52/135 2023-12-12T09:47:11.762 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 41/135 2023-12-12T09:47:11.811 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 58/135 2023-12-12T09:47:11.960 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 53/135 2023-12-12T09:47:11.970 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.7-15.el8.x86_64 42/135 2023-12-12T09:47:12.114 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 43/135 2023-12-12T09:47:12.131 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 59/135 2023-12-12T09:47:12.294 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 44/135 2023-12-12T09:47:12.313 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 60/135 2023-12-12T09:47:12.362 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 54/135 2023-12-12T09:47:12.503 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 45/135 2023-12-12T09:47:12.532 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 61/135 2023-12-12T09:47:12.671 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 55/135 2023-12-12T09:47:12.743 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 62/135 2023-12-12T09:47:12.888 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 56/135 2023-12-12T09:47:12.891 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 46/135 2023-12-12T09:47:12.951 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 63/135 2023-12-12T09:47:13.109 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 64/135 2023-12-12T09:47:13.114 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 57/135 2023-12-12T09:47:13.172 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 47/135 2023-12-12T09:47:13.388 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/135 2023-12-12T09:47:13.391 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 58/135 2023-12-12T09:47:13.420 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 65/135 2023-12-12T09:47:13.602 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 49/135 2023-12-12T09:47:13.685 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 66/135 2023-12-12T09:47:13.711 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 59/135 2023-12-12T09:47:13.841 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 50/135 2023-12-12T09:47:13.944 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 60/135 2023-12-12T09:47:13.969 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 67/135 2023-12-12T09:47:14.170 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 61/135 2023-12-12T09:47:14.173 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 68/135 2023-12-12T09:47:14.179 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 51/135 2023-12-12T09:47:14.346 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/135 2023-12-12T09:47:14.376 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 52/135 2023-12-12T09:47:14.416 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 62/135 2023-12-12T09:47:14.516 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-12T09:47:14.552 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 53/135 2023-12-12T09:47:14.581 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 63/135 2023-12-12T09:47:14.756 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/135 2023-12-12T09:47:14.797 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 64/135 2023-12-12T09:47:14.819 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 54/135 2023-12-12T09:47:14.974 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 55/135 2023-12-12T09:47:15.004 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/135 2023-12-12T09:47:15.101 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 65/135 2023-12-12T09:47:15.151 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/135 2023-12-12T09:47:15.179 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 56/135 2023-12-12T09:47:15.324 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/135 2023-12-12T09:47:15.415 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 66/135 2023-12-12T09:47:15.493 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 57/135 2023-12-12T09:47:15.522 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/135 2023-12-12T09:47:15.640 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 67/135 2023-12-12T09:47:15.708 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/135 2023-12-12T09:47:15.766 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 58/135 2023-12-12T09:47:15.870 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 68/135 2023-12-12T09:47:15.915 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/135 2023-12-12T09:47:16.051 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 59/135 2023-12-12T09:47:16.067 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/135 2023-12-12T09:47:16.068 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/135 2023-12-12T09:47:16.245 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/135 2023-12-12T09:47:16.246 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-12T09:47:16.282 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 60/135 2023-12-12T09:47:16.458 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 61/135 2023-12-12T09:47:16.462 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/135 2023-12-12T09:47:16.494 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/135 2023-12-12T09:47:16.609 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 62/135 2023-12-12T09:47:16.699 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/135 2023-12-12T09:47:16.717 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/135 2023-12-12T09:47:16.766 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : uuid-1.6.2-43.el8.x86_64 63/135 2023-12-12T09:47:16.867 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 63/135 2023-12-12T09:47:16.878 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/135 2023-12-12T09:47:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/135 2023-12-12T09:47:17.088 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/135 2023-12-12T09:47:17.094 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 64/135 2023-12-12T09:47:17.119 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/135 2023-12-12T09:47:17.294 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/135 2023-12-12T09:47:17.313 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : automake-1.16.1-7.el8.noarch 65/135 2023-12-12T09:47:17.354 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/135 2023-12-12T09:47:17.441 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 66/135 2023-12-12T09:47:17.528 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 66/135 2023-12-12T09:47:17.530 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/135 2023-12-12T09:47:17.548 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/135 2023-12-12T09:47:17.726 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : jq-1.6-3.el8.x86_64 67/135 2023-12-12T09:47:17.763 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/135 2023-12-12T09:47:17.798 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/135 2023-12-12T09:47:17.894 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 68/135 2023-12-12T09:47:17.921 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/135 2023-12-12T09:47:18.043 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 69/135 2023-12-12T09:47:18.091 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/135 2023-12-12T09:47:18.249 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-12T09:47:18.338 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/135 2023-12-12T09:47:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 71/135 2023-12-12T09:47:18.556 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/135 2023-12-12T09:47:18.741 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/135 2023-12-12T09:47:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 72/135 2023-12-12T09:47:18.879 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/135 2023-12-12T09:47:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 73/135 2023-12-12T09:47:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 74/135 2023-12-12T09:47:19.034 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/135 2023-12-12T09:47:19.167 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/135 2023-12-12T09:47:19.278 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/135 2023-12-12T09:47:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 75/135 2023-12-12T09:47:19.396 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/135 2023-12-12T09:47:19.475 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/135 2023-12-12T09:47:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 76/135 2023-12-12T09:47:19.730 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/135 2023-12-12T09:47:19.755 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 77/135 2023-12-12T09:47:19.811 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/135 2023-12-12T09:47:19.903 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 78/135 2023-12-12T09:47:20.033 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/135 2023-12-12T09:47:20.064 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 79/135 2023-12-12T09:47:20.229 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : luarocks-3.5.0-1.el8.noarch 92/135 2023-12-12T09:47:20.304 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 80/135 2023-12-12T09:47:20.410 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 93/135 2023-12-12T09:47:20.495 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 81/135 2023-12-12T09:47:20.605 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 94/135 2023-12-12T09:47:20.689 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 82/135 2023-12-12T09:47:20.923 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/135 2023-12-12T09:47:20.945 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-2.6-1.el8.noarch 83/135 2023-12-12T09:47:21.166 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/135 2023-12-12T09:47:21.246 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/135 2023-12-12T09:47:21.318 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 85/135 2023-12-12T09:47:21.509 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/135 2023-12-12T09:47:21.588 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 86/135 2023-12-12T09:47:21.852 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/135 2023-12-12T09:47:22.154 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/135 2023-12-12T09:47:22.399 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : luarocks-3.5.0-1.el8.noarch 92/135 2023-12-12T09:47:22.611 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 93/135 2023-12-12T09:47:22.790 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 87/135 2023-12-12T09:47:22.818 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 94/135 2023-12-12T09:47:23.056 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 88/135 2023-12-12T09:47:23.310 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 89/135 2023-12-12T09:47:23.616 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 90/135 2023-12-12T09:47:23.951 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 91/135 2023-12-12T09:47:24.184 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : luarocks-3.5.0-1.el8.noarch 92/135 2023-12-12T09:47:24.295 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 95/135 2023-12-12T09:47:24.368 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 93/135 2023-12-12T09:47:24.480 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 95/135 2023-12-12T09:47:24.708 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 94/135 2023-12-12T09:47:25.076 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 95/135 2023-12-12T09:47:25.365 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 96/135 2023-12-12T09:47:25.453 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 96/135 2023-12-12T09:47:25.571 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 97/135 2023-12-12T09:47:26.525 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 95/135 2023-12-12T09:47:26.751 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 95/135 2023-12-12T09:47:27.341 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 95/135 2023-12-12T09:47:27.633 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 96/135 2023-12-12T09:47:27.706 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 96/135 2023-12-12T09:47:27.867 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 97/135 2023-12-12T09:47:28.699 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 95/135 2023-12-12T09:47:28.899 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 95/135 2023-12-12T09:47:29.535 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 95/135 2023-12-12T09:47:29.836 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 96/135 2023-12-12T09:47:29.918 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 96/135 2023-12-12T09:47:30.065 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 97/135 2023-12-12T09:47:44.454 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 97/135 2023-12-12T09:47:44.454 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2023-12-12T09:47:44.454 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2023-12-12T09:47:44.455 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /dev 2023-12-12T09:47:44.455 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /run 2023-12-12T09:47:44.455 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2023-12-12T09:47:44.455 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2023-12-12T09:47:44.455 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2023-12-12T09:47:44.455 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2023-12-12T09:47:44.455 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /dev 2023-12-12T09:47:44.455 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:47:44.810 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 98/135 2023-12-12T09:47:46.422 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 98/135 2023-12-12T09:47:46.604 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 97/135 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /sys 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /proc 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /dev 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /run 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /mnt 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /var/tmp 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /home 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /tmp 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout:skipping the directory /dev 2023-12-12T09:47:46.605 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:47:46.612 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 99/135 2023-12-12T09:47:46.832 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 99/135 2023-12-12T09:47:47.007 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 98/135 2023-12-12T09:47:47.018 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 100/135 2023-12-12T09:47:47.498 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 100/135 2023-12-12T09:47:47.634 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 101/135 2023-12-12T09:47:47.651 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 101/135 2023-12-12T09:47:48.612 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 98/135 2023-12-12T09:47:48.821 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 99/135 2023-12-12T09:47:49.041 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 99/135 2023-12-12T09:47:49.247 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 100/135 2023-12-12T09:47:49.401 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 102/135 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 97/135 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /dev 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /run 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /dev 2023-12-12T09:47:49.491 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:47:49.564 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 102/135 2023-12-12T09:47:49.683 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el8 103/135 2023-12-12T09:47:49.720 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 100/135 2023-12-12T09:47:49.879 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 98/135 2023-12-12T09:47:49.906 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 101/135 2023-12-12T09:47:49.923 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 101/135 2023-12-12T09:47:49.924 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 104/135 2023-12-12T09:47:50.084 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarc 105/135 2023-12-12T09:47:50.308 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarc 105/135 2023-12-12T09:47:50.585 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noa 106/135 2023-12-12T09:47:51.545 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noa 106/135 2023-12-12T09:47:51.593 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 98/135 2023-12-12T09:47:51.643 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 102/135 2023-12-12T09:47:51.741 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 107/135 2023-12-12T09:47:51.767 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 99/135 2023-12-12T09:47:51.833 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 107/135 2023-12-12T09:47:51.840 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 102/135 2023-12-12T09:47:51.971 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el8 103/135 2023-12-12T09:47:52.001 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 99/135 2023-12-12T09:47:52.162 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el8. 108/135 2023-12-12T09:47:52.171 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 100/135 2023-12-12T09:47:52.221 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 104/135 2023-12-12T09:47:52.358 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 109/135 2023-12-12T09:47:52.395 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 109/135 2023-12-12T09:47:52.405 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarc 105/135 2023-12-12T09:47:52.583 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 110/135 2023-12-12T09:47:52.594 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 110/135 2023-12-12T09:47:52.630 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarc 105/135 2023-12-12T09:47:52.673 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 100/135 2023-12-12T09:47:52.804 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 101/135 2023-12-12T09:47:52.822 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 101/135 2023-12-12T09:47:52.954 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noa 106/135 2023-12-12T09:47:53.919 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noa 106/135 2023-12-12T09:47:53.939 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:19.0.0-96.gf1fbf3c5.el8.x86_64 111/135 2023-12-12T09:47:54.152 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 112/135 2023-12-12T09:47:54.164 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 107/135 2023-12-12T09:47:54.187 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 112/135 2023-12-12T09:47:54.255 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 107/135 2023-12-12T09:47:54.325 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 113/135 2023-12-12T09:47:54.368 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 113/135 2023-12-12T09:47:54.542 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 114/135 2023-12-12T09:47:54.613 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el8. 108/135 2023-12-12T09:47:54.693 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 102/135 2023-12-12T09:47:54.870 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 109/135 2023-12-12T09:47:54.899 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 114/135 2023-12-12T09:47:54.899 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 109/135 2023-12-12T09:47:55.003 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 102/135 2023-12-12T09:47:55.099 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 115/135 2023-12-12T09:47:55.106 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 110/135 2023-12-12T09:47:55.112 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el8 103/135 2023-12-12T09:47:55.116 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 110/135 2023-12-12T09:47:55.330 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 104/135 2023-12-12T09:47:55.495 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarc 105/135 2023-12-12T09:47:55.728 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarc 105/135 2023-12-12T09:47:55.899 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noa 106/135 2023-12-12T09:47:56.440 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-2:19.0.0-96.gf1fbf3c5.el8.x86_64 111/135 2023-12-12T09:47:56.644 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 112/135 2023-12-12T09:47:56.676 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 112/135 2023-12-12T09:47:56.863 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 113/135 2023-12-12T09:47:56.905 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 113/135 2023-12-12T09:47:56.927 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noa 106/135 2023-12-12T09:47:57.121 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 114/135 2023-12-12T09:47:57.203 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 107/135 2023-12-12T09:47:57.301 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 107/135 2023-12-12T09:47:57.480 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 114/135 2023-12-12T09:47:57.685 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el8. 108/135 2023-12-12T09:47:57.687 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 115/135 2023-12-12T09:47:57.923 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 109/135 2023-12-12T09:47:57.962 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 109/135 2023-12-12T09:47:58.136 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 110/135 2023-12-12T09:47:58.149 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 110/135 2023-12-12T09:47:59.577 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:19.0.0-96.gf1fbf3c5.el8.x86_64 111/135 2023-12-12T09:47:59.752 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 112/135 2023-12-12T09:47:59.765 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 115/135 2023-12-12T09:47:59.788 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 112/135 2023-12-12T09:47:59.936 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 113/135 2023-12-12T09:47:59.980 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 113/135 2023-12-12T09:48:00.136 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 114/135 2023-12-12T09:48:00.519 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 114/135 2023-12-12T09:48:00.612 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:19.0.0-96.gf1fbf3c5.el8.x86_64 116/135 2023-12-12T09:48:00.678 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 115/135 2023-12-12T09:48:00.960 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 117/135 2023-12-12T09:48:01.159 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 118/135 2023-12-12T09:48:01.406 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/135 2023-12-12T09:48:01.588 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/135 2023-12-12T09:48:01.778 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/135 2023-12-12T09:48:01.994 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/135 2023-12-12T09:48:02.194 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 122/135 2023-12-12T09:48:02.351 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-12T09:48:02.364 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 115/135 2023-12-12T09:48:03.274 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-test-2:19.0.0-96.gf1fbf3c5.el8.x86_64 116/135 2023-12-12T09:48:03.606 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 117/135 2023-12-12T09:48:03.827 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 118/135 2023-12-12T09:48:04.113 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/135 2023-12-12T09:48:04.278 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/135 2023-12-12T09:48:04.469 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/135 2023-12-12T09:48:04.513 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-12T09:48:04.740 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-12T09:48:04.758 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-12T09:48:04.766 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/135 2023-12-12T09:48:04.924 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 122/135 2023-12-12T09:48:04.931 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:19.0.0-96.gf1fbf3c5.el8.noarch 125/135 2023-12-12T09:48:05.073 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-12T09:48:05.248 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 126/135 2023-12-12T09:48:05.564 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 127/135 2023-12-12T09:48:05.665 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 115/135 2023-12-12T09:48:05.926 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 128/135 2023-12-12T09:48:05.949 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:19.0.0-96.gf1fbf3c5.el8.x86_64 116/135 2023-12-12T09:48:06.093 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/135 2023-12-12T09:48:06.106 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.1-9.el8.x86_64 129/135 2023-12-12T09:48:06.188 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 129/135 2023-12-12T09:48:06.300 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/135 2023-12-12T09:48:06.389 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.0.4-10.el8.x86_64 130/135 2023-12-12T09:48:06.402 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 130/135 2023-12-12T09:48:06.539 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-12T09:48:06.547 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-12T09:48:06.683 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 132/135 2023-12-12T09:48:06.683 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-12T09:48:06.892 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-12T09:48:06.893 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-12T09:48:07.104 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-12T09:48:07.258 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/135 2023-12-12T09:48:07.303 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-12T09:48:07.338 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-12T09:48:07.595 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-12T09:48:07.602 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/135 2023-12-12T09:48:07.614 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-12T09:48:07.770 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : cephfs-top-2:19.0.0-96.gf1fbf3c5.el8.noarch 125/135 2023-12-12T09:48:07.780 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 120/135 2023-12-12T09:48:07.852 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 135/135 2023-12-12T09:48:07.852 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:19.0.0-96.gf1fbf3c5.el8.x86_64 1/135 2023-12-12T09:48:07.852 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2/135 2023-12-12T09:48:07.852 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 3/135 2023-12-12T09:48:07.852 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 4/135 2023-12-12T09:48:07.852 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 5/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 6/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 7/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 8/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 9/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 10/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 11/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:19.0.0-96.gf1fbf3c5.el8.x86_64 12/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 13/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 14/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 15/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 16/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 17/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_6 18/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 19/135 2023-12-12T09:48:07.853 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el8. 20/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x8 21/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el8.x86_64 22/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:19.0.0-96.gf1fbf3c5.el8.x86_64 23/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 24/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 25/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 26/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 27/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 28/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 29/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarc 30/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noa 31/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 32/135 2023-12-12T09:48:07.854 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el8. 33/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 34/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el8 35/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 36/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 37/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:19.0.0-96.gf1fbf3c5.el8.noarch 38/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/135 2023-12-12T09:48:07.855 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/135 2023-12-12T09:48:07.856 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 73/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 74/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 75/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 76/135 2023-12-12T09:48:07.857 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 77/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 78/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 80/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 83/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 84/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 85/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 87/135 2023-12-12T09:48:07.858 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/135 2023-12-12T09:48:07.861 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 89/135 2023-12-12T09:48:07.861 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 90/135 2023-12-12T09:48:07.861 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 91/135 2023-12-12T09:48:07.861 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 92/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 93/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 94/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 95/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 97/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 98/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 99/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 100/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 101/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 102/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 103/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/135 2023-12-12T09:48:07.862 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 106/135 2023-12-12T09:48:07.863 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 107/135 2023-12-12T09:48:07.863 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 108/135 2023-12-12T09:48:07.863 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.1-7.el8.noarch 109/135 2023-12-12T09:48:07.863 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 110/135 2023-12-12T09:48:07.863 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 111/135 2023-12-12T09:48:07.863 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.1-9.el8.x86_64 112/135 2023-12-12T09:48:07.863 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 113/135 2023-12-12T09:48:07.864 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/135 2023-12-12T09:48:07.864 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 115/135 2023-12-12T09:48:07.864 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/135 2023-12-12T09:48:07.864 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 117/135 2023-12-12T09:48:07.864 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 118/135 2023-12-12T09:48:07.864 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 119/135 2023-12-12T09:48:07.865 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 120/135 2023-12-12T09:48:07.865 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 121/135 2023-12-12T09:48:07.865 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 122/135 2023-12-12T09:48:07.865 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/135 2023-12-12T09:48:07.865 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 124/135 2023-12-12T09:48:07.865 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : jq-1.6-3.el8.x86_64 125/135 2023-12-12T09:48:07.865 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 126/135 2023-12-12T09:48:07.866 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 127/135 2023-12-12T09:48:07.866 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 128/135 2023-12-12T09:48:07.866 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 129/135 2023-12-12T09:48:07.866 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 130/135 2023-12-12T09:48:07.866 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/135 2023-12-12T09:48:07.866 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 132/135 2023-12-12T09:48:07.866 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-12T09:48:07.866 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 134/135 2023-12-12T09:48:08.023 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/135 2023-12-12T09:48:08.078 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 126/135 2023-12-12T09:48:08.173 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 122/135 2023-12-12T09:48:08.313 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-12T09:48:08.425 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 127/135 2023-12-12T09:48:08.776 INFO:teuthology.orchestra.run.smithi173.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 128/135 2023-12-12T09:48:09.010 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : flex-2.6.1-9.el8.x86_64 129/135 2023-12-12T09:48:09.095 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 129/135 2023-12-12T09:48:09.311 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : bison-3.0.4-10.el8.x86_64 130/135 2023-12-12T09:48:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 130/135 2023-12-12T09:48:09.477 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-12T09:48:09.485 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 131/135 2023-12-12T09:48:09.646 INFO:teuthology.orchestra.run.smithi173.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 132/135 2023-12-12T09:48:09.646 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-12T09:48:09.872 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-12T09:48:09.872 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-12T09:48:10.151 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-12T09:48:10.393 INFO:teuthology.orchestra.run.smithi173.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-12T09:48:10.738 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-12T09:48:10.921 INFO:teuthology.orchestra.run.smithi173.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 135/135 2023-12-12T09:48:10.921 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-2:19.0.0-96.gf1fbf3c5.el8.x86_64 1/135 2023-12-12T09:48:10.921 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2/135 2023-12-12T09:48:10.921 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 3/135 2023-12-12T09:48:10.921 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 4/135 2023-12-12T09:48:10.921 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 5/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 6/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 7/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 8/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 9/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 10/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 11/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-test-2:19.0.0-96.gf1fbf3c5.el8.x86_64 12/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 13/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 14/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 15/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 16/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 17/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_6 18/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 19/135 2023-12-12T09:48:10.922 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el8. 20/135 2023-12-12T09:48:10.923 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x8 21/135 2023-12-12T09:48:10.923 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el8.x86_64 22/135 2023-12-12T09:48:10.923 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rados-2:19.0.0-96.gf1fbf3c5.el8.x86_64 23/135 2023-12-12T09:48:10.923 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 24/135 2023-12-12T09:48:10.923 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 25/135 2023-12-12T09:48:10.923 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 26/135 2023-12-12T09:48:10.923 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 27/135 2023-12-12T09:48:10.932 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 28/135 2023-12-12T09:48:10.932 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 29/135 2023-12-12T09:48:10.932 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarc 30/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noa 31/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 32/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el8. 33/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 34/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el8 35/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 36/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 37/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : cephfs-top-2:19.0.0-96.gf1fbf3c5.el8.noarch 38/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/135 2023-12-12T09:48:10.933 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/135 2023-12-12T09:48:10.934 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/135 2023-12-12T09:48:10.935 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 73/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 74/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 75/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 76/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 77/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 78/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 80/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 81/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 83/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 84/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 85/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 86/135 2023-12-12T09:48:10.936 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 87/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 89/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 90/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 91/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 92/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 93/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 94/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 95/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 97/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 98/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 99/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 100/135 2023-12-12T09:48:10.937 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 101/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 102/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 103/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 106/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 107/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 108/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : automake-1.16.1-7.el8.noarch 109/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 110/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 111/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : flex-2.6.1-9.el8.x86_64 112/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 113/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 114/135 2023-12-12T09:48:10.938 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 115/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 116/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 117/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 118/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 119/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 120/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 121/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 122/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 124/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : jq-1.6-3.el8.x86_64 125/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 126/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 127/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 128/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 129/135 2023-12-12T09:48:10.939 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 130/135 2023-12-12T09:48:10.940 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/135 2023-12-12T09:48:10.940 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 132/135 2023-12-12T09:48:10.940 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-12T09:48:10.940 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 134/135 2023-12-12T09:48:11.010 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-12T09:48:11.028 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/135 2023-12-12T09:48:11.299 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-top-2:19.0.0-96.gf1fbf3c5.el8.noarch 125/135 2023-12-12T09:48:11.594 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 126/135 2023-12-12T09:48:11.900 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 127/135 2023-12-12T09:48:12.245 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 128/135 2023-12-12T09:48:12.368 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 129/135 2023-12-12T09:48:12.379 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 129/135 2023-12-12T09:48:12.585 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 130/135 2023-12-12T09:48:12.751 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flex-2.6.1-9.el8.x86_64 131/135 2023-12-12T09:48:12.842 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 131/135 2023-12-12T09:48:12.940 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-12T09:48:12.941 INFO:teuthology.orchestra.run.smithi111.stdout:Installed products updated. 2023-12-12T09:48:13.068 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : bison-3.0.4-10.el8.x86_64 132/135 2023-12-12T09:48:13.082 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 132/135 2023-12-12T09:48:13.083 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-12T09:48:13.151 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:48:13.151 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2023-12-12T09:48:13.151 INFO:teuthology.orchestra.run.smithi111.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-12T09:48:13.151 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.151 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.151 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2023-12-12T09:48:13.151 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.1-7.el8.noarch 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.0.4-10.el8.x86_64 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.152 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.1-9.el8.x86_64 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-12T09:48:13.153 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: jq-1.6-3.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: libpq-13.5-1.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-12T09:48:13.154 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-12T09:48:13.155 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.156 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-12T09:48:13.157 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-2.6-1.el8.noarch 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-12T09:48:13.158 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-12T09:48:13.159 INFO:teuthology.orchestra.run.smithi111.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-12T09:48:13.160 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-12T09:48:13.161 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-12T09:48:13.161 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-12T09:48:13.161 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-12-12T09:48:13.161 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-12-12T09:48:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-12T09:48:13.280 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-12T09:48:13.368 DEBUG:teuthology.parallel:result is None 2023-12-12T09:48:13.550 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 134/135 2023-12-12T09:48:13.792 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 135/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:19.0.0-96.gf1fbf3c5.el8.x86_64 1/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 3/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 4/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c 5/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 6/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 7/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 8/135 2023-12-12T09:48:14.336 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 9/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 10/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 11/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:19.0.0-96.gf1fbf3c5.el8.x86_64 12/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 13/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 14/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 15/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 16/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 17/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_6 18/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 19/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el8. 20/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x8 21/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:19.0.0-96.gf1fbf3c5.el8.x86_64 22/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:19.0.0-96.gf1fbf3c5.el8.x86_64 23/135 2023-12-12T09:48:14.337 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 24/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 25/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 26/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 27/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 28/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el 29/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarc 30/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noa 31/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf 32/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el8. 33/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 34/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el8 35/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 36/135 2023-12-12T09:48:14.338 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 37/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-top-2:19.0.0-96.gf1fbf3c5.el8.noarch 38/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : luarocks-3.5.0-1.el8.noarch 42/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 43/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 44/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 45/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 46/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 47/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 48/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 49/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/135 2023-12-12T09:48:14.339 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 56/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 57/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-2.6-1.el8.noarch 58/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/135 2023-12-12T09:48:14.340 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/135 2023-12-12T09:48:14.341 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/135 2023-12-12T09:48:14.341 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/135 2023-12-12T09:48:14.341 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 71/135 2023-12-12T09:48:14.341 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 72/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 73/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 75/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 76/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 77/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : bison-3.0.4-10.el8.x86_64 79/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 80/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 81/135 2023-12-12T09:48:14.342 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 82/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : automake-1.16.1-7.el8.noarch 83/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 84/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 85/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flex-2.6.1-9.el8.x86_64 86/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 87/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 88/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 89/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 90/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 91/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 92/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 93/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 94/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 95/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 96/135 2023-12-12T09:48:14.343 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 98/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : jq-1.6-3.el8.x86_64 99/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 100/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 101/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 102/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 103/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 104/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 105/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 106/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 107/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 108/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 109/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 110/135 2023-12-12T09:48:14.344 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 111/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 112/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 114/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 115/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 116/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 117/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 118/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 119/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 121/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 122/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 123/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 124/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 125/135 2023-12-12T09:48:14.345 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 126/135 2023-12-12T09:48:14.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 127/135 2023-12-12T09:48:14.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 128/135 2023-12-12T09:48:14.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 129/135 2023-12-12T09:48:14.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 130/135 2023-12-12T09:48:14.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/135 2023-12-12T09:48:14.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 132/135 2023-12-12T09:48:14.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/135 2023-12-12T09:48:14.346 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 134/135 2023-12-12T09:48:15.909 INFO:teuthology.orchestra.run.smithi173.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-12T09:48:15.909 INFO:teuthology.orchestra.run.smithi173.stdout:Installed products updated. 2023-12-12T09:48:16.125 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:48:16.125 INFO:teuthology.orchestra.run.smithi173.stdout:Upgraded: 2023-12-12T09:48:16.125 INFO:teuthology.orchestra.run.smithi173.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-12T09:48:16.125 INFO:teuthology.orchestra.run.smithi173.stdout: librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.125 INFO:teuthology.orchestra.run.smithi173.stdout: librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.125 INFO:teuthology.orchestra.run.smithi173.stdout:Installed: 2023-12-12T09:48:16.125 INFO:teuthology.orchestra.run.smithi173.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-12T09:48:16.125 INFO:teuthology.orchestra.run.smithi173.stdout: automake-1.16.1-7.el8.noarch 2023-12-12T09:48:16.125 INFO:teuthology.orchestra.run.smithi173.stdout: bison-3.0.4-10.el8.x86_64 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.126 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-test-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-top-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: flex-2.6.1-9.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-12T09:48:16.127 INFO:teuthology.orchestra.run.smithi173.stdout: jq-1.6-3.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: libpq-13.5-1.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: librados-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.128 INFO:teuthology.orchestra.run.smithi173.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-12T09:48:16.129 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cephfs-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-12T09:48:16.130 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-12T09:48:16.131 INFO:teuthology.orchestra.run.smithi173.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-portend-2.6-1.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rados-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.132 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-12T09:48:16.133 INFO:teuthology.orchestra.run.smithi173.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-12T09:48:16.134 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.134 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.134 INFO:teuthology.orchestra.run.smithi173.stdout: rbd-nbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:16.134 INFO:teuthology.orchestra.run.smithi173.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-12T09:48:16.134 INFO:teuthology.orchestra.run.smithi173.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-12T09:48:16.134 INFO:teuthology.orchestra.run.smithi173.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-12T09:48:16.134 INFO:teuthology.orchestra.run.smithi173.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-12T09:48:16.134 INFO:teuthology.orchestra.run.smithi173.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-12T09:48:16.134 INFO:teuthology.orchestra.run.smithi173.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-12T09:48:16.135 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:48:16.135 INFO:teuthology.orchestra.run.smithi173.stdout:Complete! 2023-12-12T09:48:16.327 DEBUG:teuthology.parallel:result is None 2023-12-12T09:48:19.785 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 135/135 2023-12-12T09:48:19.785 INFO:teuthology.orchestra.run.smithi042.stdout:Installed products updated. 2023-12-12T09:48:20.020 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:48:20.020 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2023-12-12T09:48:20.020 INFO:teuthology.orchestra.run.smithi042.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-12-12T09:48:20.020 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.020 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.020 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: automake-1.16.1-7.el8.noarch 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: bison-3.0.4-10.el8.x86_64 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.021 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top-2:19.0.0-96.gf1fbf3c5.el8.noarch 2023-12-12T09:48:20.022 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: flex-2.6.1-9.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: fuse-2.9.7-15.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: jq-1.6-3.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-12T09:48:20.023 INFO:teuthology.orchestra.run.smithi042.stdout: libpq-13.5-1.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks-3.5.0-1.el8.noarch 2023-12-12T09:48:20.024 INFO:teuthology.orchestra.run.smithi042.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.15-3.el8.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-12T09:48:20.025 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-12T09:48:20.026 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.026 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.026 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.026 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-12T09:48:20.026 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-12-12T09:48:20.026 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-12T09:48:20.026 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-12T09:48:20.026 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.0.6-13.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-12T09:48:20.027 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-2.6-1.el8.noarch 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2017.2-9.el8.noarch 2023-12-12T09:48:20.028 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-12T09:48:20.029 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:19.0.0-96.gf1fbf3c5.el8.x86_64 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-12-12T09:48:20.030 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-12T09:48:20.031 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-12T09:48:20.031 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:48:20.031 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2023-12-12T09:48:20.217 DEBUG:teuthology.parallel:result is None 2023-12-12T09:48:20.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:48:20.360 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-12T09:48:20.386 INFO:teuthology.orchestra.run.smithi042.stdout:19.0.0-96.gf1fbf3c5.el8 2023-12-12T09:48:20.386 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-96.gf1fbf3c5.el8 2023-12-12T09:48:20.387 INFO:teuthology.task.install:The correct ceph version 19.0.0-96.gf1fbf3c5 is installed. 2023-12-12T09:48:20.388 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:48:20.627 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-12T09:48:20.657 INFO:teuthology.orchestra.run.smithi111.stdout:19.0.0-96.gf1fbf3c5.el8 2023-12-12T09:48:20.657 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-96.gf1fbf3c5.el8 2023-12-12T09:48:20.657 INFO:teuthology.task.install:The correct ceph version 19.0.0-96.gf1fbf3c5 is installed. 2023-12-12T09:48:20.659 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:48:20.922 DEBUG:teuthology.orchestra.run.smithi173:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-12T09:48:20.953 INFO:teuthology.orchestra.run.smithi173.stdout:19.0.0-96.gf1fbf3c5.el8 2023-12-12T09:48:20.953 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-96.gf1fbf3c5.el8 2023-12-12T09:48:20.953 INFO:teuthology.task.install:The correct ceph version 19.0.0-96.gf1fbf3c5 is installed. 2023-12-12T09:48:20.955 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-12T09:48:20.955 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:48:20.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-12T09:48:21.014 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:48:21.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-12T09:48:21.069 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:48:21.069 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-12T09:48:21.116 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-12T09:48:21.117 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:48:21.117 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2023-12-12T09:48:21.148 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-12T09:48:21.225 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:48:21.225 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2023-12-12T09:48:21.261 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-12T09:48:21.336 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:48:21.336 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2023-12-12T09:48:21.374 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-12T09:48:21.452 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-12T09:48:21.453 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:48:21.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-12T09:48:21.486 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-12T09:48:21.562 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:48:21.562 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-12T09:48:21.598 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-12T09:48:21.676 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:48:21.676 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-12T09:48:21.708 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-12T09:48:21.804 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-12T09:48:21.804 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:48:21.804 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2023-12-12T09:48:21.838 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-12T09:48:21.913 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:48:21.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/stdin-killer 2023-12-12T09:48:21.950 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-12T09:48:22.024 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:48:22.025 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/stdin-killer 2023-12-12T09:48:22.058 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-12T09:48:22.129 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-12T09:48:22.638 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-12-12T09:48:22.639 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:48:22.639 INFO:tasks.cephadm:Cluster fsid is 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:48:22.639 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-12T09:48:22.639 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.42', 'mon.b': '172.21.15.111', 'mon.c': '172.21.15.173'} 2023-12-12T09:48:22.640 INFO:tasks.cephadm:First mon is mon.a on smithi042 2023-12-12T09:48:22.640 INFO:tasks.cephadm:First mgr is x 2023-12-12T09:48:22.640 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-12T09:48:22.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname $(hostname -s) 2023-12-12T09:48:22.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo hostname $(hostname -s) 2023-12-12T09:48:22.709 DEBUG:teuthology.orchestra.run.smithi173:> sudo hostname $(hostname -s) 2023-12-12T09:48:22.742 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-12-12T09:48:22.742 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:48:22.886 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74848/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-96-gf1fbf3c5', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-96.gf1fbf3c5'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-11 05:49:21.304799', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-mchangir-testing-20231205.064351', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-12-12T09:48:23.034 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-mchangir-testing-20231205.064351, sha1 f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:48:23.035 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/x86_64/flavors/default/cephadm 2023-12-12T09:48:23.036 DEBUG:teuthology.orchestra.run.smithi042:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-12T09:48:24.401 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272055 Dec 12 09:48 /home/ubuntu/cephtest/cephadm 2023-12-12T09:48:24.401 DEBUG:teuthology.orchestra.run.smithi111:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-12T09:48:25.706 INFO:teuthology.orchestra.run.smithi111.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272055 Dec 12 09:48 /home/ubuntu/cephtest/cephadm 2023-12-12T09:48:25.706 DEBUG:teuthology.orchestra.run.smithi173:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-mchangir-testing-20231205.064351/f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-12T09:48:26.370 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-rw-r--. 1 ubuntu ubuntu 2272055 Dec 12 09:48 /home/ubuntu/cephtest/cephadm 2023-12-12T09:48:26.371 DEBUG:teuthology.orchestra.run.smithi042:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-12T09:48:26.393 DEBUG:teuthology.orchestra.run.smithi111:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-12T09:48:26.415 DEBUG:teuthology.orchestra.run.smithi173:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-12T09:48:26.456 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 on all hosts... 2023-12-12T09:48:26.456 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 pull 2023-12-12T09:48:26.459 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 pull 2023-12-12T09:48:26.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 pull 2023-12-12T09:48:30.451 INFO:teuthology.orchestra.run.smithi111.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9... 2023-12-12T09:48:31.687 INFO:teuthology.orchestra.run.smithi042.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9... 2023-12-12T09:48:31.731 INFO:teuthology.orchestra.run.smithi173.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9... 2023-12-12T09:49:00.048 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-12T09:49:00.048 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev)", 2023-12-12T09:49:00.048 INFO:teuthology.orchestra.run.smithi111.stdout: "image_id": "8cf689f388db9a4386c68a998b7e6c51ca21b0b66a804a9b803d696356b56fdc", 2023-12-12T09:49:00.048 INFO:teuthology.orchestra.run.smithi111.stdout: "repo_digests": [ 2023-12-12T09:49:00.048 INFO:teuthology.orchestra.run.smithi111.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dbb46fd2dbbe029cfba763f2b7c769e980f86b63e74be0c72e188909e20b81f3" 2023-12-12T09:49:00.048 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2023-12-12T09:49:00.048 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-12T09:49:14.882 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:49:14.882 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev)", 2023-12-12T09:49:14.882 INFO:teuthology.orchestra.run.smithi042.stdout: "image_id": "8cf689f388db9a4386c68a998b7e6c51ca21b0b66a804a9b803d696356b56fdc", 2023-12-12T09:49:14.882 INFO:teuthology.orchestra.run.smithi042.stdout: "repo_digests": [ 2023-12-12T09:49:14.882 INFO:teuthology.orchestra.run.smithi042.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dbb46fd2dbbe029cfba763f2b7c769e980f86b63e74be0c72e188909e20b81f3" 2023-12-12T09:49:14.882 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T09:49:14.882 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:49:15.339 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-12-12T09:49:15.339 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_version": "ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev)", 2023-12-12T09:49:15.340 INFO:teuthology.orchestra.run.smithi173.stdout: "image_id": "8cf689f388db9a4386c68a998b7e6c51ca21b0b66a804a9b803d696356b56fdc", 2023-12-12T09:49:15.340 INFO:teuthology.orchestra.run.smithi173.stdout: "repo_digests": [ 2023-12-12T09:49:15.340 INFO:teuthology.orchestra.run.smithi173.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dbb46fd2dbbe029cfba763f2b7c769e980f86b63e74be0c72e188909e20b81f3" 2023-12-12T09:49:15.340 INFO:teuthology.orchestra.run.smithi173.stdout: ] 2023-12-12T09:49:15.340 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-12-12T09:49:15.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph 2023-12-12T09:49:15.398 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph 2023-12-12T09:49:15.435 DEBUG:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph 2023-12-12T09:49:15.465 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /etc/ceph 2023-12-12T09:49:15.494 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /etc/ceph 2023-12-12T09:49:15.524 DEBUG:teuthology.orchestra.run.smithi173:> sudo chmod 777 /etc/ceph 2023-12-12T09:49:15.550 INFO:tasks.cephadm:Writing seed config... 2023-12-12T09:49:15.551 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-12-12T09:49:15.551 INFO:tasks.cephadm: override: [client] debug client = 20 2023-12-12T09:49:15.551 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-12-12T09:49:15.551 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-12-12T09:49:15.551 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-12-12T09:49:15.551 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-12-12T09:49:15.551 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-12-12T09:49:15.551 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-12-12T09:49:15.552 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-12-12T09:49:15.553 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-12-12T09:49:15.554 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-12-12T09:49:15.554 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-12-12T09:49:15.554 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-12-12T09:49:15.554 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-12-12T09:49:15.554 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-12-12T09:49:15.555 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:49:15.555 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-12T09:49:15.572 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 968ccaec-98d3-11ee-95a3-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-12-12T09:49:15.572 DEBUG:teuthology.orchestra.run.smithi042:mon.a> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.a.service 2023-12-12T09:49:15.615 DEBUG:teuthology.orchestra.run.smithi042:mgr.x> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.x.service 2023-12-12T09:49:15.642 INFO:journalctl@ceph.mon.a.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:49:15.643 INFO:tasks.cephadm:Bootstrapping... 2023-12-12T09:49:15.644 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 -v bootstrap --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.42 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-12T09:49:15.669 INFO:journalctl@ceph.mgr.x.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:49:15.851 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2023-12-12T09:49:15.851 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9', '-v', 'bootstrap', '--fsid', '968ccaec-98d3-11ee-95a3-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.42', '--skip-admin-label'] 2023-12-12T09:49:15.997 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.1.1 2023-12-12T09:49:15.998 INFO:teuthology.orchestra.run.smithi042.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-12-12T09:49:15.998 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying podman|docker is present... 2023-12-12T09:49:16.143 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.1.1 2023-12-12T09:49:16.144 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying lvm2 is present... 2023-12-12T09:49:16.144 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying time synchronization is in place... 2023-12-12T09:49:16.151 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-12T09:49:16.151 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-12T09:49:16.157 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-12T09:49:16.157 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2023-12-12T09:49:16.163 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2023-12-12T09:49:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2023-12-12T09:49:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2023-12-12T09:49:16.169 INFO:teuthology.orchestra.run.smithi042.stdout:Repeating the final host check... 2023-12-12T09:49:16.314 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.1.1 2023-12-12T09:49:16.314 INFO:teuthology.orchestra.run.smithi042.stdout:podman (/bin/podman) version 4.1.1 is present 2023-12-12T09:49:16.315 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl is present 2023-12-12T09:49:16.315 INFO:teuthology.orchestra.run.smithi042.stdout:lvcreate is present 2023-12-12T09:49:16.321 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-12T09:49:16.321 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-12T09:49:16.326 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-12T09:49:16.327 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2023-12-12T09:49:16.332 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2023-12-12T09:49:16.339 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2023-12-12T09:49:16.339 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2023-12-12T09:49:16.339 INFO:teuthology.orchestra.run.smithi042.stdout:Host looks OK 2023-12-12T09:49:16.339 INFO:teuthology.orchestra.run.smithi042.stdout:Cluster fsid: 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:49:16.339 INFO:teuthology.orchestra.run.smithi042.stdout:Acquiring lock 139884228942704 on /run/cephadm/968ccaec-98d3-11ee-95a3-87774f69a715.lock 2023-12-12T09:49:16.339 INFO:teuthology.orchestra.run.smithi042.stdout:Lock 139884228942704 acquired on /run/cephadm/968ccaec-98d3-11ee-95a3-87774f69a715.lock 2023-12-12T09:49:16.339 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 3300 ... 2023-12-12T09:49:16.340 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 6789 ... 2023-12-12T09:49:16.340 INFO:teuthology.orchestra.run.smithi042.stdout:Base mon IP(s) is [172.21.15.42:3300, 172.21.15.42:6789], mon addrv is [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2023-12-12T09:49:16.344 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-12-12T09:49:16.344 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.42 metric 100 2023-12-12T09:49:16.347 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-12-12T09:49:16.347 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-12-12T09:49:16.347 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1758sec hoplimit 64 pref medium 2023-12-12T09:49:16.349 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-12T09:49:16.349 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-12-12T09:49:16.350 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-12T09:49:16.350 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 3: ens1f0: mtu 1500 state UP qlen 1000 2023-12-12T09:49:16.350 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1440/64 scope link 2023-12-12T09:49:16.350 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-12T09:49:16.350 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2023-12-12T09:49:16.350 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2023-12-12T09:49:16.351 INFO:teuthology.orchestra.run.smithi042.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-12-12T09:49:16.351 INFO:teuthology.orchestra.run.smithi042.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-12T09:49:16.352 INFO:teuthology.orchestra.run.smithi042.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9... 2023-12-12T09:49:16.797 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 8cf689f388db9a4386c68a998b7e6c51ca21b0b66a804a9b803d696356b56fdc 2023-12-12T09:49:16.797 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9... 2023-12-12T09:49:16.797 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Getting image source signatures 2023-12-12T09:49:16.797 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:48620857745a2578a33397824134e62955869ca630a4abf0f0c74967e2c37293 2023-12-12T09:49:16.797 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:d3e436ac93e4b5a6d94c7ac8024df331e66e67275836276615dc044fd3cbf83a 2023-12-12T09:49:16.798 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying config sha256:8cf689f388db9a4386c68a998b7e6c51ca21b0b66a804a9b803d696356b56fdc 2023-12-12T09:49:16.798 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Writing manifest to image destination 2023-12-12T09:49:16.798 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Storing signatures 2023-12-12T09:49:17.649 INFO:teuthology.orchestra.run.smithi042.stdout:ceph: stdout ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev) 2023-12-12T09:49:17.650 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph version: ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev) 2023-12-12T09:49:17.650 INFO:teuthology.orchestra.run.smithi042.stdout:Extracting ceph user uid/gid from container image... 2023-12-12T09:49:18.400 INFO:teuthology.orchestra.run.smithi042.stdout:stat: stdout 167 167 2023-12-12T09:49:18.400 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial keys... 2023-12-12T09:49:19.651 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCfLHhlZuW6ERAAg13BKnMnX/uEpIBsR+ZnQQ== 2023-12-12T09:49:20.398 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCgLHhlURsuAxAAJTbqmd55DqVE9OMVT9dUHw== 2023-12-12T09:49:21.148 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCgLHhl6EyfLhAA+FVooicbu8gQUiSmSC4Ulw== 2023-12-12T09:49:21.148 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial monmap... 2023-12-12T09:49:21.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-12T09:49:21.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-12-12T09:49:21.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:49:21.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-12T09:49:21.894 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool for a [v2:172.21.15.42:3300,v1:172.21.15.42:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-12T09:49:21.894 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = pacific 2023-12-12T09:49:21.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: set fsid to 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:49:21.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-12T09:49:21.895 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:49:21.895 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mon... 2023-12-12T09:49:22.639 INFO:teuthology.orchestra.run.smithi042.stdout:create mon.a on 2023-12-12T09:49:22.758 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-12-12T09:49:22.853 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-12-12T09:49:22.976 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-968ccaec-98d3-11ee-95a3-87774f69a715.target → /etc/systemd/system/ceph-968ccaec-98d3-11ee-95a3-87774f69a715.target. 2023-12-12T09:49:22.976 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-968ccaec-98d3-11ee-95a3-87774f69a715.target → /etc/systemd/system/ceph-968ccaec-98d3-11ee-95a3-87774f69a715.target. 2023-12-12T09:49:23.169 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.a 2023-12-12T09:49:23.169 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.a.service: Unit ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.a.service not loaded. 2023-12-12T09:49:23.261 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-968ccaec-98d3-11ee-95a3-87774f69a715.target.wants/ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.a.service → /etc/systemd/system/ceph-968ccaec-98d3-11ee-95a3-87774f69a715@.service. 2023-12-12T09:49:23.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:23 smithi042 systemd[1]: Starting Ceph mon.a for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:49:23.859 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:23 smithi042 podman[98355]: 2023-12-12T09:49:23.859 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:23 smithi042 bash[98355]: 810967644c703e2d14de2f20a0781c5f812a9d9027f19755d92afa61d286ce3b 2023-12-12T09:49:23.859 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:23 smithi042 ceph-mon[98396]: mkfs 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:49:23.859 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:23 smithi042 ceph-mon[98396]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-12T09:49:23.871 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-12T09:49:23.872 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2023-12-12T09:49:23.879 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-12T09:49:23.879 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2023-12-12T09:49:23.879 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2023-12-12T09:49:23.879 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2023-12-12T09:49:23.880 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon to start... 2023-12-12T09:49:23.880 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon... 2023-12-12T09:49:24.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:23 smithi042 systemd[1]: Started Ceph mon.a for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:49:24.832 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout cluster: 2023-12-12T09:49:24.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout id: 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:49:24.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-12-12T09:49:24.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:24.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout services: 2023-12-12T09:49:24.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.680221s) 2023-12-12T09:49:24.833 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout data: 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pgs: 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.346+0000 7f4482ef4700 1 Processor -- start 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.346+0000 7f4482ef4700 1 -- start start 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.346+0000 7f4482ef4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c107700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.346+0000 7f4482ef4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c107cd0 con 0x7f447c1072e0 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.346+0000 7f4480c90700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c107700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:24.834 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.346+0000 7f4480c90700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c107700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60296/0 (socket says 172.21.15.42:60296) 2023-12-12T09:49:24.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.346+0000 7f4480c90700 1 -- 172.21.15.42:0/201573111 learned_addr learned my addr 172.21.15.42:0/201573111 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:24.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.347+0000 7f4480c90700 1 -- 172.21.15.42:0/201573111 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c105da0 con 0x7f447c1072e0 2023-12-12T09:49:24.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.347+0000 7f4480c90700 1 --2- 172.21.15.42:0/201573111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c107700 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44700098a0 tx=0x7f4470009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8338df6188c1c8d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:24.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.347+0000 7f447b7fe700 1 -- 172.21.15.42:0/201573111 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4470004160 con 0x7f447c1072e0 2023-12-12T09:49:24.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.347+0000 7f447b7fe700 1 -- 172.21.15.42:0/201573111 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44700042c0 con 0x7f447c1072e0 2023-12-12T09:49:24.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.347+0000 7f447b7fe700 1 -- 172.21.15.42:0/201573111 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44700045a0 con 0x7f447c1072e0 2023-12-12T09:49:24.835 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 -- 172.21.15.42:0/201573111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 msgr2=0x7f447c107700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 --2- 172.21.15.42:0/201573111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c107700 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f44700098a0 tx=0x7f4470009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 -- 172.21.15.42:0/201573111 shutdown_connections 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 --2- 172.21.15.42:0/201573111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c107700 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 -- 172.21.15.42:0/201573111 >> 172.21.15.42:0/201573111 conn(0x7f447c100970 msgr2=0x7f447c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 -- 172.21.15.42:0/201573111 shutdown_connections 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 -- 172.21.15.42:0/201573111 wait complete. 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 Processor -- start 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 -- start start 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c18e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.348+0000 7f4482ef4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c18ed00 con 0x7f447c1072e0 2023-12-12T09:49:24.836 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4480c90700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c18e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4480c90700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c18e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60298/0 (socket says 172.21.15.42:60298) 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4480c90700 1 -- 172.21.15.42:0/1465791372 learned_addr learned my addr 172.21.15.42:0/1465791372 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4480c90700 1 -- 172.21.15.42:0/1465791372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4470009550 con 0x7f447c1072e0 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4480c90700 1 --2- 172.21.15.42:0/1465791372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c18e7c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f447c073f70 tx=0x7f4470004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4479ffb700 1 -- 172.21.15.42:0/1465791372 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4470011630 con 0x7f447c1072e0 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4479ffb700 1 -- 172.21.15.42:0/1465791372 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470011790 con 0x7f447c1072e0 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4482ef4700 1 -- 172.21.15.42:0/1465791372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c18cf40 con 0x7f447c1072e0 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4479ffb700 1 -- 172.21.15.42:0/1465791372 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4470011a30 con 0x7f447c1072e0 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.349+0000 7f4482ef4700 1 -- 172.21.15.42:0/1465791372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c18d3e0 con 0x7f447c1072e0 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.350+0000 7f4479ffb700 1 -- 172.21.15.42:0/1465791372 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4470022070 con 0x7f447c1072e0 2023-12-12T09:49:24.837 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.350+0000 7f4479ffb700 1 -- 172.21.15.42:0/1465791372 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f447001c450 con 0x7f447c1072e0 2023-12-12T09:49:24.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.350+0000 7f4482ef4700 1 -- 172.21.15.42:0/1465791372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c18ddf0 con 0x7f447c1072e0 2023-12-12T09:49:24.838 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.353+0000 7f4479ffb700 1 -- 172.21.15.42:0/1465791372 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f447001b450 con 0x7f447c1072e0 2023-12-12T09:49:24.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.486+0000 7f4482ef4700 1 -- 172.21.15.42:0/1465791372 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f447c106110 con 0x7f447c1072e0 2023-12-12T09:49:24.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.486+0000 7f4479ffb700 1 -- 172.21.15.42:0/1465791372 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f447001b5f0 con 0x7f447c1072e0 2023-12-12T09:49:24.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.487+0000 7f4482ef4700 1 -- 172.21.15.42:0/1465791372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 msgr2=0x7f447c18e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:24.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.487+0000 7f4482ef4700 1 --2- 172.21.15.42:0/1465791372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c18e7c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f447c073f70 tx=0x7f4470004750 comp rx=0 tx=0).stop 2023-12-12T09:49:24.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.487+0000 7f4482ef4700 1 -- 172.21.15.42:0/1465791372 shutdown_connections 2023-12-12T09:49:24.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.487+0000 7f4482ef4700 1 --2- 172.21.15.42:0/1465791372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f447c1072e0 0x7f447c18e7c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:24.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.487+0000 7f4482ef4700 1 -- 172.21.15.42:0/1465791372 >> 172.21.15.42:0/1465791372 conn(0x7f447c100970 msgr2=0x7f447c191040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:24.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.487+0000 7f4482ef4700 1 -- 172.21.15.42:0/1465791372 shutdown_connections 2023-12-12T09:49:24.844 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:24.487+0000 7f4482ef4700 1 -- 172.21.15.42:0/1465791372 wait complete. 2023-12-12T09:49:24.845 INFO:teuthology.orchestra.run.smithi042.stdout:mon is available 2023-12-12T09:49:24.845 INFO:teuthology.orchestra.run.smithi042.stdout:Assimilating anything we can from ceph.conf... 2023-12-12T09:49:25.074 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:24 smithi042 ceph-mon[98396]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-12T09:49:25.074 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:24 smithi042 ceph-mon[98396]: monmap e1: 1 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:49:25.074 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:24 smithi042 ceph-mon[98396]: fsmap 2023-12-12T09:49:25.075 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:24 smithi042 ceph-mon[98396]: osdmap e1: 0 total, 0 up, 0 in 2023-12-12T09:49:25.075 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:24 smithi042 ceph-mon[98396]: mgrmap e1: no daemons active 2023-12-12T09:49:25.075 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:24 smithi042 ceph-mon[98396]: from='client.? 172.21.15.42:0/1465791372' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-12T09:49:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:25.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-12T09:49:25.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.347+0000 7fa32d3cb700 1 Processor -- start 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.347+0000 7fa32d3cb700 1 -- start start 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.348+0000 7fa32d3cb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328109c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.348+0000 7fa32d3cb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32807ada0 con 0x7fa328107860 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.348+0000 7fa326ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.348+0000 7fa326ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328109c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60300/0 (socket says 172.21.15.42:60300) 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.348+0000 7fa326ffd700 1 -- 172.21.15.42:0/930702674 learned_addr learned my addr 172.21.15.42:0/930702674 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.348+0000 7fa326ffd700 1 -- 172.21.15.42:0/930702674 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32807aee0 con 0x7fa328107860 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.348+0000 7fa326ffd700 1 --2- 172.21.15.42:0/930702674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328109c50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa3180098a0 tx=0x7fa318009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ae0a9233b91246 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.348+0000 7fa325ffb700 1 -- 172.21.15.42:0/930702674 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa318004160 con 0x7fa328107860 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.349+0000 7fa325ffb700 1 -- 172.21.15.42:0/930702674 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3180042c0 con 0x7fa328107860 2023-12-12T09:49:25.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.349+0000 7fa325ffb700 1 -- 172.21.15.42:0/930702674 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3180045a0 con 0x7fa328107860 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.349+0000 7fa32d3cb700 1 -- 172.21.15.42:0/930702674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 msgr2=0x7fa328109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.349+0000 7fa32d3cb700 1 --2- 172.21.15.42:0/930702674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328109c50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa3180098a0 tx=0x7fa318009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.349+0000 7fa32d3cb700 1 -- 172.21.15.42:0/930702674 shutdown_connections 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.349+0000 7fa32d3cb700 1 --2- 172.21.15.42:0/930702674 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328109c50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.349+0000 7fa32d3cb700 1 -- 172.21.15.42:0/930702674 >> 172.21.15.42:0/930702674 conn(0x7fa3281011d0 msgr2=0x7fa328103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.349+0000 7fa32d3cb700 1 -- 172.21.15.42:0/930702674 shutdown_connections 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.349+0000 7fa32d3cb700 1 -- 172.21.15.42:0/930702674 wait complete. 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.350+0000 7fa32d3cb700 1 Processor -- start 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.350+0000 7fa32d3cb700 1 -- start start 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.350+0000 7fa32d3cb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328107090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.350+0000 7fa32d3cb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3281075d0 con 0x7fa328107860 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.350+0000 7fa326ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.350+0000 7fa326ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328107090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60302/0 (socket says 172.21.15.42:60302) 2023-12-12T09:49:25.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.350+0000 7fa326ffd700 1 -- 172.21.15.42:0/3363628805 learned_addr learned my addr 172.21.15.42:0/3363628805 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.351+0000 7fa326ffd700 1 -- 172.21.15.42:0/3363628805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa318009550 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.351+0000 7fa326ffd700 1 --2- 172.21.15.42:0/3363628805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328107090 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa32807a1e0 tx=0x7fa318004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.351+0000 7fa30ffff700 1 -- 172.21.15.42:0/3363628805 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa318011630 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.351+0000 7fa30ffff700 1 -- 172.21.15.42:0/3363628805 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa318011790 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.351+0000 7fa32d3cb700 1 -- 172.21.15.42:0/3363628805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa328105810 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.351+0000 7fa30ffff700 1 -- 172.21.15.42:0/3363628805 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa318011a30 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.351+0000 7fa32d3cb700 1 -- 172.21.15.42:0/3363628805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa328105cb0 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.351+0000 7fa30ffff700 1 -- 172.21.15.42:0/3363628805 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa318022070 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.351+0000 7fa30ffff700 1 -- 172.21.15.42:0/3363628805 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa31801c450 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.352+0000 7fa32d3cb700 1 -- 172.21.15.42:0/3363628805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa328068600 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.354+0000 7fa30ffff700 1 -- 172.21.15.42:0/3363628805 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fa31801b450 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.483+0000 7fa32d3cb700 1 -- 172.21.15.42:0/3363628805 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa32810a370 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.485+0000 7fa30ffff700 1 -- 172.21.15.42:0/3363628805 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fa31801b5f0 con 0x7fa328107860 2023-12-12T09:49:25.884 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.486+0000 7fa30ffff700 1 -- 172.21.15.42:0/3363628805 <== mon.0 v2:172.21.15.42:3300/0 8 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa31801b790 con 0x7fa328107860 2023-12-12T09:49:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.487+0000 7fa32d3cb700 1 -- 172.21.15.42:0/3363628805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 msgr2=0x7fa328107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.487+0000 7fa32d3cb700 1 --2- 172.21.15.42:0/3363628805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328107090 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa32807a1e0 tx=0x7fa318004750 comp rx=0 tx=0).stop 2023-12-12T09:49:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.487+0000 7fa32d3cb700 1 -- 172.21.15.42:0/3363628805 shutdown_connections 2023-12-12T09:49:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.487+0000 7fa32d3cb700 1 --2- 172.21.15.42:0/3363628805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa328107860 0x7fa328107090 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.487+0000 7fa32d3cb700 1 -- 172.21.15.42:0/3363628805 >> 172.21.15.42:0/3363628805 conn(0x7fa3281011d0 msgr2=0x7fa3281034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.487+0000 7fa32d3cb700 1 -- 172.21.15.42:0/3363628805 shutdown_connections 2023-12-12T09:49:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:25.487+0000 7fa32d3cb700 1 -- 172.21.15.42:0/3363628805 wait complete. 2023-12-12T09:49:25.885 INFO:teuthology.orchestra.run.smithi042.stdout:Generating new minimal ceph.conf... 2023-12-12T09:49:26.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:25 smithi042 ceph-mon[98396]: from='client.? 172.21.15.42:0/3363628805' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-12T09:49:26.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:25 smithi042 ceph-mon[98396]: from='client.? 172.21.15.42:0/3363628805' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-12T09:49:26.822 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.323+0000 7ff817116700 1 Processor -- start 2023-12-12T09:49:26.822 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.323+0000 7ff817116700 1 -- start start 2023-12-12T09:49:26.822 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.323+0000 7ff817116700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810109bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:26.822 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.323+0000 7ff817116700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81007ae40 con 0x7ff8101077d0 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.323+0000 7ff814eb2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810109bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.323+0000 7ff814eb2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810109bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60304/0 (socket says 172.21.15.42:60304) 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.323+0000 7ff814eb2700 1 -- 172.21.15.42:0/3110001019 learned_addr learned my addr 172.21.15.42:0/3110001019 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.324+0000 7ff814eb2700 1 -- 172.21.15.42:0/3110001019 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81010a100 con 0x7ff8101077d0 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.324+0000 7ff814eb2700 1 --2- 172.21.15.42:0/3110001019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810109bc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098a0 tx=0x7ff804009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05fe051eb833dc0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.324+0000 7ff80f7fe700 1 -- 172.21.15.42:0/3110001019 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff804004160 con 0x7ff8101077d0 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.324+0000 7ff80f7fe700 1 -- 172.21.15.42:0/3110001019 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8040042c0 con 0x7ff8101077d0 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.324+0000 7ff80f7fe700 1 -- 172.21.15.42:0/3110001019 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff804011a00 con 0x7ff8101077d0 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.325+0000 7ff817116700 1 -- 172.21.15.42:0/3110001019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 msgr2=0x7ff810109bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.325+0000 7ff817116700 1 --2- 172.21.15.42:0/3110001019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810109bc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098a0 tx=0x7ff804009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.325+0000 7ff817116700 1 -- 172.21.15.42:0/3110001019 shutdown_connections 2023-12-12T09:49:26.823 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.325+0000 7ff817116700 1 --2- 172.21.15.42:0/3110001019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810109bc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.325+0000 7ff817116700 1 -- 172.21.15.42:0/3110001019 >> 172.21.15.42:0/3110001019 conn(0x7ff8101011c0 msgr2=0x7ff810103620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.325+0000 7ff817116700 1 -- 172.21.15.42:0/3110001019 shutdown_connections 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.325+0000 7ff817116700 1 -- 172.21.15.42:0/3110001019 wait complete. 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.325+0000 7ff817116700 1 Processor -- start 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.325+0000 7ff817116700 1 -- start start 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff817116700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810107000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff817116700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff810107540 con 0x7ff8101077d0 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff814eb2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff814eb2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810107000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60306/0 (socket says 172.21.15.42:60306) 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff814eb2700 1 -- 172.21.15.42:0/2966150786 learned_addr learned my addr 172.21.15.42:0/2966150786 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff814eb2700 1 -- 172.21.15.42:0/2966150786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804009550 con 0x7ff8101077d0 2023-12-12T09:49:26.824 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff814eb2700 1 --2- 172.21.15.42:0/2966150786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810107000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff804009fb0 tx=0x7ff804011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff80dffb700 1 -- 172.21.15.42:0/2966150786 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff804003a40 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff80dffb700 1 -- 172.21.15.42:0/2966150786 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff804003ba0 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff817116700 1 -- 172.21.15.42:0/2966150786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff810105780 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.326+0000 7ff80dffb700 1 -- 172.21.15.42:0/2966150786 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff80401a750 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.327+0000 7ff817116700 1 -- 172.21.15.42:0/2966150786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff810105c20 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.327+0000 7ff80dffb700 1 -- 172.21.15.42:0/2966150786 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff80401acd0 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.327+0000 7ff80dffb700 1 -- 172.21.15.42:0/2966150786 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff8040193c0 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.328+0000 7ff817116700 1 -- 172.21.15.42:0/2966150786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff810068600 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.330+0000 7ff80dffb700 1 -- 172.21.15.42:0/2966150786 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff80402de40 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.458+0000 7ff817116700 1 -- 172.21.15.42:0/2966150786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff81010b110 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.459+0000 7ff80dffb700 1 -- 172.21.15.42:0/2966150786 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7ff804024c10 con 0x7ff8101077d0 2023-12-12T09:49:26.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.460+0000 7ff817116700 1 -- 172.21.15.42:0/2966150786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 msgr2=0x7ff810107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:26.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.460+0000 7ff817116700 1 --2- 172.21.15.42:0/2966150786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810107000 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff804009fb0 tx=0x7ff804011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:26.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.460+0000 7ff817116700 1 -- 172.21.15.42:0/2966150786 shutdown_connections 2023-12-12T09:49:26.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.460+0000 7ff817116700 1 --2- 172.21.15.42:0/2966150786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8101077d0 0x7ff810107000 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:26.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.460+0000 7ff817116700 1 -- 172.21.15.42:0/2966150786 >> 172.21.15.42:0/2966150786 conn(0x7ff8101011c0 msgr2=0x7ff810101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:26.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.460+0000 7ff817116700 1 -- 172.21.15.42:0/2966150786 shutdown_connections 2023-12-12T09:49:26.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:26.460+0000 7ff817116700 1 -- 172.21.15.42:0/2966150786 wait complete. 2023-12-12T09:49:26.826 INFO:teuthology.orchestra.run.smithi042.stdout:Restarting the monitor... 2023-12-12T09:49:27.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:26 smithi042 ceph-mon[98396]: from='client.? 172.21.15.42:0/2966150786' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:49:27.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:26 smithi042 systemd[1]: Stopping Ceph mon.a for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:49:27.469 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:27 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-a[98371]: 2023-12-12T09:49:27.153+0000 7fe0377b0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T09:49:27.469 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:27 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-a[98371]: 2023-12-12T09:49:27.153+0000 7fe0377b0700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-12T09:49:27.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:27 smithi042 bash[98729]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-a 2023-12-12T09:49:27.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:27 smithi042 systemd[1]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.a.service: Succeeded. 2023-12-12T09:49:27.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:27 smithi042 systemd[1]: Stopped Ceph mon.a for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:49:27.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:27 smithi042 systemd[1]: Starting Ceph mon.a for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:49:28.198 INFO:teuthology.orchestra.run.smithi042.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-12-12T09:49:28.234 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:27 smithi042 podman[98854]: 2023-12-12T09:49:28.234 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: set uid:gid to 167:167 (ceph:ceph) 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev), process ceph-mon, pid 7 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: pidfile_write: ignore empty --pid-file 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 bash[98854]: 9b13c8f8a6fa0999206958693b539e359a60929ad9908bd559f8c7e5dc3dd246 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: load: jerasure load: lrc 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: RocksDB version: 7.9.2 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Git sha 0 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Compile date 2023-12-11 04:34:03 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: DB SUMMARY 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: DB Session ID: MRBHQ8R3Y3C3HJZP8AFI 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: CURRENT file: CURRENT 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: IDENTITY file: IDENTITY 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 98829 ; 2023-12-12T09:49:28.235 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.error_if_exists: 0 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.create_if_missing: 0 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.paranoid_checks: 1 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.env: 0x55a05c65fa60 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.fs: PosixFileSystem 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.info_log: 0x55a05e739740 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_file_opening_threads: 16 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.statistics: (nil) 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.use_fsync: 0 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_log_file_size: 0 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-12T09:49:28.236 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.keep_log_file_num: 1000 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.recycle_log_file_num: 0 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.allow_fallocate: 1 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.allow_mmap_reads: 0 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.allow_mmap_writes: 0 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.use_direct_reads: 0 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.create_missing_column_families: 0 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.db_log_dir: 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.wal_dir: 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-12T09:49:28.237 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.advise_random_on_open: 1 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.db_write_buffer_size: 0 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.write_buffer_manager: 0x55a05e6ef360 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.rate_limiter: (nil) 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.wal_recovery_mode: 2 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.enable_thread_tracking: 0 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.enable_pipelined_write: 0 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.unordered_write: 0 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-12T09:49:28.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.row_cache: None 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.wal_filter: None 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.allow_ingest_behind: 0 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.two_write_queues: 0 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.manual_wal_flush: 0 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.wal_compression: 0 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.atomic_flush: 0 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-12T09:49:28.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.log_readahead_size: 0 2023-12-12T09:49:28.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-12T09:49:28.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.best_efforts_recovery: 0 2023-12-12T09:49:28.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-12T09:49:28.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-12T09:49:28.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.allow_data_in_errors: 0 2023-12-12T09:49:28.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.db_host_id: __hostname__ 2023-12-12T09:49:28.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-12T09:49:28.241 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_background_jobs: 2 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_background_compactions: -1 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_subcompactions: 1 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_total_wal_size: 0 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_open_files: -1 2023-12-12T09:49:28.242 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bytes_per_sync: 0 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_readahead_size: 0 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_background_flushes: -1 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Compression algorithms supported: 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: kZSTD supported: 0 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: kXpressCompression supported: 0 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: kLZ4HCCompression supported: 1 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: kZlibCompression supported: 1 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: kSnappyCompression supported: 1 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: kLZ4Compression supported: 1 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: kBZip2Compression supported: 0 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-12T09:49:28.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.merge_operator: 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_filter: None 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_filter_factory: None 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.sst_partitioner_factory: None 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a05e739860) 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks: 1 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_top_level_index_and_filter: 1 2023-12-12T09:49:28.244 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_type: 0 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_index_type: 0 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_shortening: 1 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: checksum: 4 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: no_block_cache: 0 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache: 0x55a05e78f090 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_name: BinnedLRUCache 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_options: 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: capacity : 536870912 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_shard_bits : 4 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: strict_capacity_limit : 0 2023-12-12T09:49:28.245 INFO:journalctl@ceph.mon.a.smithi042.stdout: high_pri_pool_ratio: 0.000 2023-12-12T09:49:28.246 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_compressed: (nil) 2023-12-12T09:49:28.246 INFO:journalctl@ceph.mon.a.smithi042.stdout: persistent_cache: (nil) 2023-12-12T09:49:28.246 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size: 4096 2023-12-12T09:49:28.246 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size_deviation: 10 2023-12-12T09:49:28.246 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_restart_interval: 16 2023-12-12T09:49:28.246 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_block_restart_interval: 1 2023-12-12T09:49:28.246 INFO:journalctl@ceph.mon.a.smithi042.stdout: metadata_block_size: 4096 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: partition_filters: 0 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: use_delta_encoding: 1 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: filter_policy: bloomfilter 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: whole_key_filtering: 1 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: verify_compression: 0 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: read_amp_bytes_per_bit: 0 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: format_version: 5 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_index_compression: 1 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_align: 0 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: max_auto_readahead_size: 262144 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: prepopulate_block_cache: 0 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: initial_auto_readahead_size: 8192 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_file_reads_for_auto_readahead: 2 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.write_buffer_size: 33554432 2023-12-12T09:49:28.247 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_write_buffer_number: 2 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression: NoCompression 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression: Disabled 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.prefix_extractor: nullptr 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.num_levels: 7 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-12T09:49:28.248 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression_opts.level: 32767 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression_opts.strategy: 0 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-12T09:49:28.249 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-12T09:49:28.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-12T09:49:28.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-12T09:49:28.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression_opts.enabled: false 2023-12-12T09:49:28.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-12T09:49:28.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-12T09:49:28.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-12T09:49:28.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-12T09:49:28.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.target_file_size_base: 67108864 2023-12-12T09:49:28.250 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.arena_block_size: 1048576 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-12T09:49:28.251 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-12T09:49:28.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.disable_auto_compactions: 0 2023-12-12T09:49:28.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-12T09:49:28.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-12T09:49:28.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-12T09:49:28.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-12T09:49:28.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-12T09:49:28.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-12T09:49:28.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-12T09:49:28.252 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.table_properties_collectors: 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.inplace_update_support: 0 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.bloom_locality: 0 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.max_successive_merges: 0 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.paranoid_file_checks: 0 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.force_consistency_checks: 1 2023-12-12T09:49:28.253 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.report_bg_io_stats: 0 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.ttl: 2592000 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.enable_blob_files: false 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.min_blob_size: 0 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.blob_file_size: 268435456 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-12T09:49:28.254 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.blob_file_starting_level: 0 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ceb42a50-6c92-4a19-b062-e13b2ab1082a 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702374568157757, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702374568159287, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94080, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 92085, "index_size": 277, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13642, "raw_average_key_size": 49, "raw_value_size": 84638, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702374568, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ceb42a50-6c92-4a19-b062-e13b2ab1082a", "db_session_id": "MRBHQ8R3Y3C3HJZP8AFI", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702374568159389, "job": 1, "event": "recovery_finished"} 2023-12-12T09:49:28.255 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a05e82a000 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: DB pointer 0x55a05e816000 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** DB Stats ** 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-12T09:49:28.256 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: L0 2/0 93.71 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: Sum 2/0 93.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 102.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-12T09:49:28.257 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 102.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-12T09:49:28.258 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative compaction: 0.00 GB write, 19.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval compaction: 0.00 GB write, 19.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache BinnedLRUCache@0x55a05e78f090#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: starting mon.a rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mon.a@-1(???) e1 preinit fsid 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mon.a@-1(???).mds e1 new map 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mon.a@-1(???).mds e1 print_map 2023-12-12T09:49:28.259 INFO:journalctl@ceph.mon.a.smithi042.stdout: e1 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout: legacy client fscid: -1 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout: No filesystems configured 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 systemd[1]: Started Ceph mon.a for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: monmap e1: 1 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: fsmap 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: osdmap e1: 0 total, 0 up, 0 in 2023-12-12T09:49:28.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:28 smithi042 ceph-mon[98894]: mgrmap e1: no daemons active 2023-12-12T09:49:29.347 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.762+0000 7f503b5ad700 1 Processor -- start 2023-12-12T09:49:29.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.762+0000 7f503b5ad700 1 -- start start 2023-12-12T09:49:29.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.762+0000 7f503b5ad700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f5034107110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:29.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.762+0000 7f503b5ad700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50341076e0 con 0x7f5034106cf0 2023-12-12T09:49:29.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.762+0000 7f5039349700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f5034107110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:29.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.763+0000 7f5039349700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f5034107110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60308/0 (socket says 172.21.15.42:60308) 2023-12-12T09:49:29.352 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.763+0000 7f5039349700 1 -- 172.21.15.42:0/416796569 learned_addr learned my addr 172.21.15.42:0/416796569 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.763+0000 7f5039349700 1 -- 172.21.15.42:0/416796569 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5034107ef0 con 0x7f5034106cf0 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.763+0000 7f5039349700 1 --2- 172.21.15.42:0/416796569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f5034107110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f50300098a0 tx=0x7f5030009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7681d99533e22785 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.763+0000 7f502bfff700 1 -- 172.21.15.42:0/416796569 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5030004160 con 0x7f5034106cf0 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.763+0000 7f502bfff700 1 -- 172.21.15.42:0/416796569 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f50300042c0 con 0x7f5034106cf0 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.763+0000 7f502bfff700 1 -- 172.21.15.42:0/416796569 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5030011a00 con 0x7f5034106cf0 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.764+0000 7f503b5ad700 1 -- 172.21.15.42:0/416796569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 msgr2=0x7f5034107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.764+0000 7f503b5ad700 1 --2- 172.21.15.42:0/416796569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f5034107110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f50300098a0 tx=0x7f5030009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.764+0000 7f503b5ad700 1 -- 172.21.15.42:0/416796569 shutdown_connections 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.764+0000 7f503b5ad700 1 --2- 172.21.15.42:0/416796569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f5034107110 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.764+0000 7f503b5ad700 1 -- 172.21.15.42:0/416796569 >> 172.21.15.42:0/416796569 conn(0x7f50341022d0 msgr2=0x7f50341046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:29.353 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.764+0000 7f503b5ad700 1 -- 172.21.15.42:0/416796569 shutdown_connections 2023-12-12T09:49:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.764+0000 7f503b5ad700 1 -- 172.21.15.42:0/416796569 wait complete. 2023-12-12T09:49:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.765+0000 7f503b5ad700 1 Processor -- start 2023-12-12T09:49:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.765+0000 7f503b5ad700 1 -- start start 2023-12-12T09:49:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.765+0000 7f503b5ad700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f503419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.765+0000 7f503b5ad700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503419b060 con 0x7f5034106cf0 2023-12-12T09:49:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.765+0000 7f5039349700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f503419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.765+0000 7f5039349700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f503419ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60310/0 (socket says 172.21.15.42:60310) 2023-12-12T09:49:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.765+0000 7f5039349700 1 -- 172.21.15.42:0/543262203 learned_addr learned my addr 172.21.15.42:0/543262203 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:29.354 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.765+0000 7f5039349700 1 -- 172.21.15.42:0/543262203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5030009550 con 0x7f5034106cf0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.765+0000 7f5039349700 1 --2- 172.21.15.42:0/543262203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f503419ab20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5030009fb0 tx=0x7f5030011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.766+0000 7f502a7fc700 1 -- 172.21.15.42:0/543262203 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5030003a40 con 0x7f5034106cf0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.766+0000 7f502a7fc700 1 -- 172.21.15.42:0/543262203 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5030003ba0 con 0x7f5034106cf0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.766+0000 7f503b5ad700 1 -- 172.21.15.42:0/543262203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f503419b260 con 0x7f5034106cf0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.766+0000 7f502a7fc700 1 -- 172.21.15.42:0/543262203 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f503001a750 con 0x7f5034106cf0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.766+0000 7f503b5ad700 1 -- 172.21.15.42:0/543262203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f503419bd70 con 0x7f5034106cf0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.766+0000 7f502a7fc700 1 -- 172.21.15.42:0/543262203 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f503001ac80 con 0x7f5034106cf0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.766+0000 7f502a7fc700 1 -- 172.21.15.42:0/543262203 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f50300193c0 con 0x7f5034106cf0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.767+0000 7f503b5ad700 1 -- 172.21.15.42:0/543262203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f503419b700 con 0x7f5034106cf0 2023-12-12T09:49:29.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.769+0000 7f502a7fc700 1 -- 172.21.15.42:0/543262203 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f503419b700 con 0x7f5034106cf0 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.896+0000 7f503b5ad700 1 -- 172.21.15.42:0/543262203 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f5034062c00 con 0x7f5034106cf0 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.898+0000 7f502a7fc700 1 -- 172.21.15.42:0/543262203 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f503002de30 con 0x7f5034106cf0 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.899+0000 7f503b5ad700 1 -- 172.21.15.42:0/543262203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 msgr2=0x7f503419ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.899+0000 7f503b5ad700 1 --2- 172.21.15.42:0/543262203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f503419ab20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5030009fb0 tx=0x7f5030011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.899+0000 7f503b5ad700 1 -- 172.21.15.42:0/543262203 shutdown_connections 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.899+0000 7f503b5ad700 1 --2- 172.21.15.42:0/543262203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5034106cf0 0x7f503419ab20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.899+0000 7f503b5ad700 1 -- 172.21.15.42:0/543262203 >> 172.21.15.42:0/543262203 conn(0x7f50341022d0 msgr2=0x7f5034102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.899+0000 7f503b5ad700 1 -- 172.21.15.42:0/543262203 shutdown_connections 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:28.899+0000 7f503b5ad700 1 -- 172.21.15.42:0/543262203 wait complete. 2023-12-12T09:49:29.356 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote config to /etc/ceph/ceph.conf 2023-12-12T09:49:29.377 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-12T09:49:29.378 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mgr... 2023-12-12T09:49:29.378 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:9283 ... 2023-12-12T09:49:29.378 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:8765 ... 2023-12-12T09:49:29.586 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.x 2023-12-12T09:49:29.586 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.x.service: Unit ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.x.service not loaded. 2023-12-12T09:49:29.680 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-968ccaec-98d3-11ee-95a3-87774f69a715.target.wants/ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-968ccaec-98d3-11ee-95a3-87774f69a715@.service. 2023-12-12T09:49:29.900 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:29 smithi042 systemd[1]: Starting Ceph mgr.x for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:49:30.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:29 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/543262203' entity='client.admin' 2023-12-12T09:49:30.155 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:29 smithi042 podman[99119]: 2023-12-12T09:49:30.291 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-12T09:49:30.291 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2023-12-12T09:49:30.300 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-12T09:49:30.300 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2023-12-12T09:49:30.300 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2023-12-12T09:49:30.300 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2023-12-12T09:49:30.308 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-12T09:49:30.308 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2023-12-12T09:49:30.316 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-12T09:49:30.316 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2023-12-12T09:49:30.316 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2023-12-12T09:49:30.316 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-12-12T09:49:30.317 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr to start... 2023-12-12T09:49:30.317 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr... 2023-12-12T09:49:30.568 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:30 smithi042 bash[99119]: 13967c32f53ea4e4eb3ff876674e67f9db350bc2c456fe5ab68ca8193ef2cd20 2023-12-12T09:49:30.568 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:30 smithi042 systemd[1]: Started Ceph mgr.x for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:49:30.568 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:30.369+0000 7f6ef0c1e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:49:30.569 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:30.465+0000 7f6ef0c1e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:49:30.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:30.577+0000 7f6ef0c1e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:49:30.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:30.693+0000 7f6ef0c1e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:49:31.358 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:31 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/869930625' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-12T09:49:31.359 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:30.907+0000 7f6ef0c1e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:49:31.373 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:31.373 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2023-12-12T09:49:31.373 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:49:31.374 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2023-12-12T09:49:31.374 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-12T09:49:31.374 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-12T09:49:31.374 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-12T09:49:31.374 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:31.374 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-12T09:49:31.374 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-12T09:49:31.374 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2023-12-12T09:49:31.378 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:31.378 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-12T09:49:31.378 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2023-12-12T09:49:31.378 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:31.378 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-12-12T09:49:31.378 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-12T09:49:31.378 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:31.378 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-12T09:49:31.378 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-12T09:49:31.379 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-12T09:49:31.380 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-12T09:49:31.380 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-12T09:49:31.380 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-12T09:49:31.380 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-12T09:49:31.380 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-12T09:49:31.380 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-12T09:49:31.380 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-12T09:49:31.380 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-12T09:49:31.381 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2023-12-12T09:49:23.808712+0000", 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:31.382 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.875+0000 7f84d66c6700 1 Processor -- start 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.875+0000 7f84d66c6700 1 -- start start 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.875+0000 7f84d66c6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d0105a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.875+0000 7f84d66c6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d007ade0 con 0x7f84d0105610 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.876+0000 7f84cffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d0105a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.876+0000 7f84cffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d0105a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60316/0 (socket says 172.21.15.42:60316) 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.876+0000 7f84cffff700 1 -- 172.21.15.42:0/4158559210 learned_addr learned my addr 172.21.15.42:0/4158559210 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.876+0000 7f84cffff700 1 -- 172.21.15.42:0/4158559210 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d0105f70 con 0x7f84d0105610 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.876+0000 7f84cffff700 1 --2- 172.21.15.42:0/4158559210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d0105a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098a0 tx=0x7f84c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e38e4d479be1a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:31.383 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.877+0000 7f84ceffd700 1 -- 172.21.15.42:0/4158559210 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84c0004160 con 0x7f84d0105610 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.877+0000 7f84ceffd700 1 -- 172.21.15.42:0/4158559210 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84c00042c0 con 0x7f84d0105610 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.877+0000 7f84ceffd700 1 -- 172.21.15.42:0/4158559210 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84c0011a00 con 0x7f84d0105610 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.877+0000 7f84d66c6700 1 -- 172.21.15.42:0/4158559210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 msgr2=0x7f84d0105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.877+0000 7f84d66c6700 1 --2- 172.21.15.42:0/4158559210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d0105a30 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098a0 tx=0x7f84c0009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.877+0000 7f84d66c6700 1 -- 172.21.15.42:0/4158559210 shutdown_connections 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.877+0000 7f84d66c6700 1 --2- 172.21.15.42:0/4158559210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d0105a30 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.877+0000 7f84d66c6700 1 -- 172.21.15.42:0/4158559210 >> 172.21.15.42:0/4158559210 conn(0x7f84d01011d0 msgr2=0x7f84d0103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.877+0000 7f84d66c6700 1 -- 172.21.15.42:0/4158559210 shutdown_connections 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.878+0000 7f84d66c6700 1 -- 172.21.15.42:0/4158559210 wait complete. 2023-12-12T09:49:31.384 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.878+0000 7f84d66c6700 1 Processor -- start 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.878+0000 7f84d66c6700 1 -- start start 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.878+0000 7f84d66c6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d019cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84d66c6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d019d160 con 0x7f84d0105610 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84cffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d019cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84cffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d019cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60318/0 (socket says 172.21.15.42:60318) 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84cffff700 1 -- 172.21.15.42:0/869930625 learned_addr learned my addr 172.21.15.42:0/869930625 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84cffff700 1 -- 172.21.15.42:0/869930625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c0009550 con 0x7f84d0105610 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84cffff700 1 --2- 172.21.15.42:0/869930625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d019cc20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009fb0 tx=0x7f84c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84cd7fa700 1 -- 172.21.15.42:0/869930625 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84c0003a40 con 0x7f84d0105610 2023-12-12T09:49:31.385 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84cd7fa700 1 -- 172.21.15.42:0/869930625 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f84c0003ba0 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84d66c6700 1 -- 172.21.15.42:0/869930625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d019d360 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84cd7fa700 1 -- 172.21.15.42:0/869930625 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84c001a750 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.879+0000 7f84d66c6700 1 -- 172.21.15.42:0/869930625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d019d800 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.880+0000 7f84cd7fa700 1 -- 172.21.15.42:0/869930625 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f84c001ac80 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.880+0000 7f84cd7fa700 1 -- 172.21.15.42:0/869930625 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f84c00193c0 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.881+0000 7f84d66c6700 1 -- 172.21.15.42:0/869930625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d0196990 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:30.885+0000 7f84cd7fa700 1 -- 172.21.15.42:0/869930625 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f84c0024b60 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:31.019+0000 7f84d66c6700 1 -- 172.21.15.42:0/869930625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f84d0106ed0 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:31.020+0000 7f84cd7fa700 1 -- 172.21.15.42:0/869930625 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f84c0024d00 con 0x7f84d0105610 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:31.021+0000 7f84d66c6700 1 -- 172.21.15.42:0/869930625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 msgr2=0x7f84d019cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:31.386 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:31.021+0000 7f84d66c6700 1 --2- 172.21.15.42:0/869930625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d019cc20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f84c0009fb0 tx=0x7f84c0011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:31.387 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:31.022+0000 7f84d66c6700 1 -- 172.21.15.42:0/869930625 shutdown_connections 2023-12-12T09:49:31.387 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:31.022+0000 7f84d66c6700 1 --2- 172.21.15.42:0/869930625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d0105610 0x7f84d019cc20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:31.387 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:31.022+0000 7f84d66c6700 1 -- 172.21.15.42:0/869930625 >> 172.21.15.42:0/869930625 conn(0x7f84d01011d0 msgr2=0x7f84d0193ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:31.387 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:31.022+0000 7f84d66c6700 1 -- 172.21.15.42:0/869930625 shutdown_connections 2023-12-12T09:49:31.387 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:31.022+0000 7f84d66c6700 1 -- 172.21.15.42:0/869930625 wait complete. 2023-12-12T09:49:31.387 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (1/15)... 2023-12-12T09:49:31.652 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:31.398+0000 7f6ef0c1e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:49:31.653 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:31.550+0000 7f6ef0c1e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:49:32.152 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:31.657+0000 7f6ef0c1e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:49:32.152 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:31.772+0000 7f6ef0c1e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:49:32.653 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:32 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:32.282+0000 7f6ef0c1e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:49:33.402 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:33.078+0000 7f6ef0c1e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:49:33.402 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:33.176+0000 7f6ef0c1e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:49:34.287 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:34 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/220780578' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-12T09:49:34.288 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:34.175+0000 7f6ef0c1e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:49:34.422 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:34.422 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2023-12-12T09:49:34.422 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:49:34.422 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-12T09:49:34.423 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-12T09:49:34.424 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-12T09:49:34.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-12T09:49:34.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-12T09:49:34.433 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:34.434 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:34.435 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2023-12-12T09:49:23.808712+0000", 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.823+0000 7f1b57e68700 1 Processor -- start 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.823+0000 7f1b57e68700 1 -- start start 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.824+0000 7f1b57e68700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b50106050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.824+0000 7f1b57e68700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b50106620 con 0x7f1b50105c30 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.824+0000 7f1b55c04700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b50106050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.824+0000 7f1b55c04700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b50106050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60320/0 (socket says 172.21.15.42:60320) 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.824+0000 7f1b55c04700 1 -- 172.21.15.42:0/4002854896 learned_addr learned my addr 172.21.15.42:0/4002854896 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:34.436 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.824+0000 7f1b55c04700 1 -- 172.21.15.42:0/4002854896 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50106e30 con 0x7f1b50105c30 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.824+0000 7f1b55c04700 1 --2- 172.21.15.42:0/4002854896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b50106050 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0098a0 tx=0x7f1b4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05b47ccb02328b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.825+0000 7f1b54c02700 1 -- 172.21.15.42:0/4002854896 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b4c004160 con 0x7f1b50105c30 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.825+0000 7f1b54c02700 1 -- 172.21.15.42:0/4002854896 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b4c0042c0 con 0x7f1b50105c30 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.825+0000 7f1b54c02700 1 -- 172.21.15.42:0/4002854896 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b4c011a00 con 0x7f1b50105c30 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.826+0000 7f1b57e68700 1 -- 172.21.15.42:0/4002854896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 msgr2=0x7f1b50106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.826+0000 7f1b57e68700 1 --2- 172.21.15.42:0/4002854896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b50106050 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0098a0 tx=0x7f1b4c009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.826+0000 7f1b57e68700 1 -- 172.21.15.42:0/4002854896 shutdown_connections 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.826+0000 7f1b57e68700 1 --2- 172.21.15.42:0/4002854896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b50106050 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.826+0000 7f1b57e68700 1 -- 172.21.15.42:0/4002854896 >> 172.21.15.42:0/4002854896 conn(0x7f1b501011d0 msgr2=0x7f1b50103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.826+0000 7f1b57e68700 1 -- 172.21.15.42:0/4002854896 shutdown_connections 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.826+0000 7f1b57e68700 1 -- 172.21.15.42:0/4002854896 wait complete. 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.827+0000 7f1b57e68700 1 Processor -- start 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.827+0000 7f1b57e68700 1 -- start start 2023-12-12T09:49:34.437 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.827+0000 7f1b57e68700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b5007a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:34.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.827+0000 7f1b57e68700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5007aec0 con 0x7f1b50105c30 2023-12-12T09:49:34.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.827+0000 7f1b55c04700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b5007a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:34.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.827+0000 7f1b55c04700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b5007a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60322/0 (socket says 172.21.15.42:60322) 2023-12-12T09:49:34.438 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.827+0000 7f1b55c04700 1 -- 172.21.15.42:0/220780578 learned_addr learned my addr 172.21.15.42:0/220780578 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.828+0000 7f1b55c04700 1 -- 172.21.15.42:0/220780578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c009550 con 0x7f1b50105c30 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.828+0000 7f1b55c04700 1 --2- 172.21.15.42:0/220780578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b5007a980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009fb0 tx=0x7f1b4c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.828+0000 7f1b46ffd700 1 -- 172.21.15.42:0/220780578 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b4c003a40 con 0x7f1b50105c30 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.828+0000 7f1b57e68700 1 -- 172.21.15.42:0/220780578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5019d7b0 con 0x7f1b50105c30 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.828+0000 7f1b46ffd700 1 -- 172.21.15.42:0/220780578 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1b4c003ba0 con 0x7f1b50105c30 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.828+0000 7f1b46ffd700 1 -- 172.21.15.42:0/220780578 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b4c01a750 con 0x7f1b50105c30 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.828+0000 7f1b57e68700 1 -- 172.21.15.42:0/220780578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5019dbf0 con 0x7f1b50105c30 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.828+0000 7f1b46ffd700 1 -- 172.21.15.42:0/220780578 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1b4c01acd0 con 0x7f1b50105c30 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.828+0000 7f1b46ffd700 1 -- 172.21.15.42:0/220780578 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1b4c0193c0 con 0x7f1b50105c30 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.829+0000 7f1b57e68700 1 -- 172.21.15.42:0/220780578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b50068600 con 0x7f1b50105c30 2023-12-12T09:49:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.831+0000 7f1b46ffd700 1 -- 172.21.15.42:0/220780578 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f1b4c02de40 con 0x7f1b50105c30 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.966+0000 7f1b57e68700 1 -- 172.21.15.42:0/220780578 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1b50109fc0 con 0x7f1b50105c30 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.967+0000 7f1b46ffd700 1 -- 172.21.15.42:0/220780578 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f1b4c024b60 con 0x7f1b50105c30 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.968+0000 7f1b57e68700 1 -- 172.21.15.42:0/220780578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 msgr2=0x7f1b5007a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.968+0000 7f1b57e68700 1 --2- 172.21.15.42:0/220780578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b5007a980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009fb0 tx=0x7f1b4c011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.968+0000 7f1b57e68700 1 -- 172.21.15.42:0/220780578 shutdown_connections 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.968+0000 7f1b57e68700 1 --2- 172.21.15.42:0/220780578 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b50105c30 0x7f1b5007a980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.968+0000 7f1b57e68700 1 -- 172.21.15.42:0/220780578 >> 172.21.15.42:0/220780578 conn(0x7f1b501011d0 msgr2=0x7f1b50101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.968+0000 7f1b57e68700 1 -- 172.21.15.42:0/220780578 shutdown_connections 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:33.968+0000 7f1b57e68700 1 -- 172.21.15.42:0/220780578 wait complete. 2023-12-12T09:49:34.440 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (2/15)... 2023-12-12T09:49:34.652 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:34.285+0000 7f6ef0c1e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:49:34.653 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:34.508+0000 7f6ef0c1e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:49:35.296 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:35.006+0000 7f6ef0c1e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:49:35.296 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:35.111+0000 7f6ef0c1e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:49:35.625 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:35.294+0000 7f6ef0c1e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:49:35.902 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:35.623+0000 7f6ef0c1e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:49:36.707 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:36.451+0000 7f6ef0c1e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:49:36.707 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:36.612+0000 7f6ef0c1e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:49:37.123 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:36.864+0000 7f6ef0c1e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:49:37.123 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:37.002+0000 7f6ef0c1e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:49:37.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:37 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3680072347' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-12T09:49:37.403 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:37.284+0000 7f6ef0c1e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:49:37.473 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:37.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-12T09:49:37.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2023-12-12T09:49:37.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:37.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-12-12T09:49:37.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-12T09:49:37.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:37.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-12T09:49:37.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-12T09:49:37.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:37.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-12T09:49:37.486 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-12T09:49:37.487 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:37.488 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-12T09:49:37.489 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2023-12-12T09:49:23.808712+0000", 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.928+0000 7f3251b37700 1 Processor -- start 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.929+0000 7f3251b37700 1 -- start start 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.929+0000 7f3251b37700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c106050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.929+0000 7f3251b37700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c106620 con 0x7f324c105c30 2023-12-12T09:49:37.490 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.929+0000 7f324b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c106050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.929+0000 7f324b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c106050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60324/0 (socket says 172.21.15.42:60324) 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.929+0000 7f324b7fe700 1 -- 172.21.15.42:0/2109717227 learned_addr learned my addr 172.21.15.42:0/2109717227 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.929+0000 7f324b7fe700 1 -- 172.21.15.42:0/2109717227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c106e30 con 0x7f324c105c30 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.930+0000 7f324b7fe700 1 --2- 172.21.15.42:0/2109717227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c106050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f323c0098a0 tx=0x7f323c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4a50ff60f301e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.930+0000 7f324a7fc700 1 -- 172.21.15.42:0/2109717227 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f323c004160 con 0x7f324c105c30 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.930+0000 7f324a7fc700 1 -- 172.21.15.42:0/2109717227 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f323c0042c0 con 0x7f324c105c30 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.930+0000 7f324a7fc700 1 -- 172.21.15.42:0/2109717227 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f323c011a00 con 0x7f324c105c30 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.931+0000 7f3251b37700 1 -- 172.21.15.42:0/2109717227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 msgr2=0x7f324c106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.931+0000 7f3251b37700 1 --2- 172.21.15.42:0/2109717227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c106050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f323c0098a0 tx=0x7f323c009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.931+0000 7f3251b37700 1 -- 172.21.15.42:0/2109717227 shutdown_connections 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.931+0000 7f3251b37700 1 --2- 172.21.15.42:0/2109717227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c106050 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:37.491 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.931+0000 7f3251b37700 1 -- 172.21.15.42:0/2109717227 >> 172.21.15.42:0/2109717227 conn(0x7f324c1011d0 msgr2=0x7f324c103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.931+0000 7f3251b37700 1 -- 172.21.15.42:0/2109717227 shutdown_connections 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.931+0000 7f3251b37700 1 -- 172.21.15.42:0/2109717227 wait complete. 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.932+0000 7f3251b37700 1 Processor -- start 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.932+0000 7f3251b37700 1 -- start start 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.932+0000 7f3251b37700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c07a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.932+0000 7f3251b37700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c07aec0 con 0x7f324c105c30 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.932+0000 7f324b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c07a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.932+0000 7f324b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c07a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60326/0 (socket says 172.21.15.42:60326) 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.932+0000 7f324b7fe700 1 -- 172.21.15.42:0/3680072347 learned_addr learned my addr 172.21.15.42:0/3680072347 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.932+0000 7f324b7fe700 1 -- 172.21.15.42:0/3680072347 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c009550 con 0x7f324c105c30 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.933+0000 7f324b7fe700 1 --2- 172.21.15.42:0/3680072347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c07a980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f323c009130 tx=0x7f323c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.933+0000 7f3248ff9700 1 -- 172.21.15.42:0/3680072347 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f323c003ab0 con 0x7f324c105c30 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.933+0000 7f3248ff9700 1 -- 172.21.15.42:0/3680072347 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f323c003c10 con 0x7f324c105c30 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.933+0000 7f3251b37700 1 -- 172.21.15.42:0/3680072347 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c19d7b0 con 0x7f324c105c30 2023-12-12T09:49:37.492 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.933+0000 7f3248ff9700 1 -- 172.21.15.42:0/3680072347 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f323c0247d0 con 0x7f324c105c30 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.933+0000 7f3251b37700 1 -- 172.21.15.42:0/3680072347 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c19dbf0 con 0x7f324c105c30 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.933+0000 7f3248ff9700 1 -- 172.21.15.42:0/3680072347 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f323c024d50 con 0x7f324c105c30 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.933+0000 7f3248ff9700 1 -- 172.21.15.42:0/3680072347 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f323c0193c0 con 0x7f324c105c30 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.934+0000 7f3251b37700 1 -- 172.21.15.42:0/3680072347 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324c109ff0 con 0x7f324c105c30 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:36.936+0000 7f3248ff9700 1 -- 172.21.15.42:0/3680072347 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f323c01abb0 con 0x7f324c105c30 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:37.070+0000 7f3251b37700 1 -- 172.21.15.42:0/3680072347 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f324c10a1e0 con 0x7f324c105c30 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:37.070+0000 7f3248ff9700 1 -- 172.21.15.42:0/3680072347 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f323c035090 con 0x7f324c105c30 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:37.071+0000 7f3251b37700 1 -- 172.21.15.42:0/3680072347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 msgr2=0x7f324c07a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:37.071+0000 7f3251b37700 1 --2- 172.21.15.42:0/3680072347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c07a980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f323c009130 tx=0x7f323c011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:37.072+0000 7f3251b37700 1 -- 172.21.15.42:0/3680072347 shutdown_connections 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:37.072+0000 7f3251b37700 1 --2- 172.21.15.42:0/3680072347 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f324c105c30 0x7f324c07a980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:37.072+0000 7f3251b37700 1 -- 172.21.15.42:0/3680072347 >> 172.21.15.42:0/3680072347 conn(0x7f324c1011d0 msgr2=0x7f324c101d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:37.072+0000 7f3251b37700 1 -- 172.21.15.42:0/3680072347 shutdown_connections 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:37.072+0000 7f3251b37700 1 -- 172.21.15.42:0/3680072347 wait complete. 2023-12-12T09:49:37.493 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (3/15)... 2023-12-12T09:49:38.652 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:38 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:38.329+0000 7f6ef0c1e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:49:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: Activating manager daemon x 2023-12-12T09:49:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: mgrmap e2: x(active, starting, since 0.00324669s) 2023-12-12T09:49:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-12T09:49:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-12T09:49:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-12T09:49:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:49:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-12T09:49:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: Manager daemon x is now available 2023-12-12T09:49:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-12T09:49:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-12T09:49:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' 2023-12-12T09:49:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' 2023-12-12T09:49:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:39 smithi042 ceph-mon[98894]: from='mgr.14100 172.21.15.42:0/2943532839' entity='mgr.x' 2023-12-12T09:49:40.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:40 smithi042 ceph-mon[98894]: mgrmap e3: x(active, since 1.007s) 2023-12-12T09:49:42.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:42 smithi042 ceph-mon[98894]: mgrmap e4: x(active, since 3s) 2023-12-12T09:49:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:42 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/4288301271' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-12T09:49:44.941 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:44.942 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-12T09:49:44.943 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-12T09:49:44.944 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-12T09:49:44.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-12T09:49:44.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-12T09:49:44.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-12T09:49:44.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:44.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:44.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2023-12-12T09:49:23.808712+0000", 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-12T09:49:44.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.385+0000 7f2e73cec700 1 Processor -- start 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.386+0000 7f2e73cec700 1 -- start start 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.386+0000 7f2e73cec700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c109be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.386+0000 7f2e73cec700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c07ae20 con 0x7f2e6c1077f0 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.386+0000 7f2e71a88700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.386+0000 7f2e71a88700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60352/0 (socket says 172.21.15.42:60352) 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.386+0000 7f2e71a88700 1 -- 172.21.15.42:0/1090619030 learned_addr learned my addr 172.21.15.42:0/1090619030 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.386+0000 7f2e71a88700 1 -- 172.21.15.42:0/1090619030 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6c10a120 con 0x7f2e6c1077f0 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.386+0000 7f2e71a88700 1 --2- 172.21.15.42:0/1090619030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c109be0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e680098a0 tx=0x7f2e68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0620abf27fe047 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e70a86700 1 -- 172.21.15.42:0/1090619030 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e68004160 con 0x7f2e6c1077f0 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e70a86700 1 -- 172.21.15.42:0/1090619030 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e680042c0 con 0x7f2e6c1077f0 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e70a86700 1 -- 172.21.15.42:0/1090619030 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e68011a00 con 0x7f2e6c1077f0 2023-12-12T09:49:44.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e73cec700 1 -- 172.21.15.42:0/1090619030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 msgr2=0x7f2e6c109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e73cec700 1 --2- 172.21.15.42:0/1090619030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c109be0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e680098a0 tx=0x7f2e68009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e73cec700 1 -- 172.21.15.42:0/1090619030 shutdown_connections 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e73cec700 1 --2- 172.21.15.42:0/1090619030 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c109be0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e73cec700 1 -- 172.21.15.42:0/1090619030 >> 172.21.15.42:0/1090619030 conn(0x7f2e6c1011c0 msgr2=0x7f2e6c103620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e73cec700 1 -- 172.21.15.42:0/1090619030 shutdown_connections 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.387+0000 7f2e73cec700 1 -- 172.21.15.42:0/1090619030 wait complete. 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.388+0000 7f2e73cec700 1 Processor -- start 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.388+0000 7f2e73cec700 1 -- start start 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.388+0000 7f2e73cec700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.388+0000 7f2e73cec700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6c19af40 con 0x7f2e6c1077f0 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.388+0000 7f2e71a88700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.388+0000 7f2e71a88700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60354/0 (socket says 172.21.15.42:60354) 2023-12-12T09:49:44.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.388+0000 7f2e71a88700 1 -- 172.21.15.42:0/4288301271 learned_addr learned my addr 172.21.15.42:0/4288301271 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.389+0000 7f2e71a88700 1 -- 172.21.15.42:0/4288301271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e68009550 con 0x7f2e6c1077f0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.389+0000 7f2e71a88700 1 --2- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c19aa00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009130 tx=0x7f2e68011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.389+0000 7f2e62ffd700 1 -- 172.21.15.42:0/4288301271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e68003b70 con 0x7f2e6c1077f0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.389+0000 7f2e62ffd700 1 -- 172.21.15.42:0/4288301271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2e68003cd0 con 0x7f2e6c1077f0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.389+0000 7f2e73cec700 1 -- 172.21.15.42:0/4288301271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6c19b140 con 0x7f2e6c1077f0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.389+0000 7f2e62ffd700 1 -- 172.21.15.42:0/4288301271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e680197d0 con 0x7f2e6c1077f0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.389+0000 7f2e73cec700 1 -- 172.21.15.42:0/4288301271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6c19b5e0 con 0x7f2e6c1077f0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.390+0000 7f2e62ffd700 1 -- 172.21.15.42:0/4288301271 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f2e6801abb0 con 0x7f2e6c1077f0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.390+0000 7f2e62ffd700 1 --2- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f2e58038820 0x7f2e5803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.390+0000 7f2e60ff9700 1 -- 172.21.15.42:0/4288301271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e540052f0 con 0x7f2e6c1077f0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.390+0000 7f2e62ffd700 1 -- 172.21.15.42:0/4288301271 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2e6804cad0 con 0x7f2e6c1077f0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.390+0000 7f2e71287700 1 --2- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f2e58038820 0x7f2e5803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:44.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.391+0000 7f2e71287700 1 --2- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f2e58038820 0x7f2e5803ace0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c006fd0 tx=0x7f2e5c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:44.958 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.396+0000 7f2e62ffd700 1 -- 172.21.15.42:0/4288301271 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2e6801a430 con 0x7f2e6c1077f0 2023-12-12T09:49:44.958 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.926+0000 7f2e60ff9700 1 -- 172.21.15.42:0/4288301271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2e54005160 con 0x7f2e6c1077f0 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.926+0000 7f2e62ffd700 1 -- 172.21.15.42:0/4288301271 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f2e68023370 con 0x7f2e6c1077f0 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.928+0000 7f2e60ff9700 1 -- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f2e58038820 msgr2=0x7f2e5803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.928+0000 7f2e60ff9700 1 --2- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f2e58038820 0x7f2e5803ace0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2e5c006fd0 tx=0x7f2e5c006e40 comp rx=0 tx=0).stop 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.928+0000 7f2e60ff9700 1 -- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 msgr2=0x7f2e6c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.928+0000 7f2e60ff9700 1 --2- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c19aa00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2e68009130 tx=0x7f2e68011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.928+0000 7f2e60ff9700 1 -- 172.21.15.42:0/4288301271 shutdown_connections 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.929+0000 7f2e60ff9700 1 --2- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f2e58038820 0x7f2e5803ace0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.929+0000 7f2e60ff9700 1 --2- 172.21.15.42:0/4288301271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e6c1077f0 0x7f2e6c19aa00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.929+0000 7f2e60ff9700 1 -- 172.21.15.42:0/4288301271 >> 172.21.15.42:0/4288301271 conn(0x7f2e6c1011c0 msgr2=0x7f2e6c101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.929+0000 7f2e60ff9700 1 -- 172.21.15.42:0/4288301271 shutdown_connections 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:41.929+0000 7f2e60ff9700 1 -- 172.21.15.42:0/4288301271 wait complete. 2023-12-12T09:49:44.959 INFO:teuthology.orchestra.run.smithi042.stdout:mgr is available 2023-12-12T09:49:47.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:47 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2431769073' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-12T09:49:49.010 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-12T09:49:49.011 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.181+0000 7f49fbb31700 1 Processor -- start 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.181+0000 7f49fbb31700 1 -- start start 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.181+0000 7f49fbb31700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f4107100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.181+0000 7f49fbb31700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f41076d0 con 0x7f49f4106ce0 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.181+0000 7f49f98cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f4107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.181+0000 7f49f98cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f4107100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60358/0 (socket says 172.21.15.42:60358) 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.181+0000 7f49f98cd700 1 -- 172.21.15.42:0/50415963 learned_addr learned my addr 172.21.15.42:0/50415963 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.182+0000 7f49f98cd700 1 -- 172.21.15.42:0/50415963 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f4107ee0 con 0x7f49f4106ce0 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.182+0000 7f49f98cd700 1 --2- 172.21.15.42:0/50415963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f4107100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49f00098a0 tx=0x7f49f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52e1b0dd72f779e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.182+0000 7f49f88cb700 1 -- 172.21.15.42:0/50415963 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49f0004160 con 0x7f49f4106ce0 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.182+0000 7f49f88cb700 1 -- 172.21.15.42:0/50415963 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49f00042c0 con 0x7f49f4106ce0 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.182+0000 7f49f88cb700 1 -- 172.21.15.42:0/50415963 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49f0011a00 con 0x7f49f4106ce0 2023-12-12T09:49:49.012 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.183+0000 7f49fbb31700 1 -- 172.21.15.42:0/50415963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 msgr2=0x7f49f4107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.183+0000 7f49fbb31700 1 --2- 172.21.15.42:0/50415963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f4107100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49f00098a0 tx=0x7f49f0009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.183+0000 7f49fbb31700 1 -- 172.21.15.42:0/50415963 shutdown_connections 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.183+0000 7f49fbb31700 1 --2- 172.21.15.42:0/50415963 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f4107100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.183+0000 7f49fbb31700 1 -- 172.21.15.42:0/50415963 >> 172.21.15.42:0/50415963 conn(0x7f49f41022e0 msgr2=0x7f49f4104700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.183+0000 7f49fbb31700 1 -- 172.21.15.42:0/50415963 shutdown_connections 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.183+0000 7f49fbb31700 1 -- 172.21.15.42:0/50415963 wait complete. 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.183+0000 7f49fbb31700 1 Processor -- start 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.183+0000 7f49fbb31700 1 -- start start 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49fbb31700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f407cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49fbb31700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f407d010 con 0x7f49f4106ce0 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49f98cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f407cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49f98cd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f407cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60360/0 (socket says 172.21.15.42:60360) 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49f98cd700 1 -- 172.21.15.42:0/2431769073 learned_addr learned my addr 172.21.15.42:0/2431769073 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49f98cd700 1 -- 172.21.15.42:0/2431769073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f0009550 con 0x7f49f4106ce0 2023-12-12T09:49:49.013 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49f98cd700 1 --2- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f407cad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009130 tx=0x7f49f0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49eaffd700 1 -- 172.21.15.42:0/2431769073 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49f0003ab0 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49eaffd700 1 -- 172.21.15.42:0/2431769073 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49f0003c10 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f407b250 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49eaffd700 1 -- 172.21.15.42:0/2431769073 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49f00247d0 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.184+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f407b6f0 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.185+0000 7f49eaffd700 1 -- 172.21.15.42:0/2431769073 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f49f001abb0 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.186+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f4068600 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.186+0000 7f49eaffd700 1 --2- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f49e0038820 0x7f49e003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.186+0000 7f49eaffd700 1 -- 172.21.15.42:0/2431769073 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f49f004c540 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.186+0000 7f49f90cc700 1 --2- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f49e0038820 0x7f49e003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.186+0000 7f49f90cc700 1 --2- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f49e0038820 0x7f49e003ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.191+0000 7f49eaffd700 1 -- 172.21.15.42:0/2431769073 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f49f0018710 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.553+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f49f410b0d0 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.554+0000 7f49eaffd700 1 -- 172.21.15.42:0/2431769073 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f49f0018350 con 0x7f49f4106ce0 2023-12-12T09:49:49.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f49e0038820 msgr2=0x7f49e003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 --2- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f49e0038820 0x7f49e003ace0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f49e4006fd0 tx=0x7f49e4006e40 comp rx=0 tx=0).stop 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 msgr2=0x7f49f407cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 --2- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f407cad0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f49f0009130 tx=0x7f49f0011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 shutdown_connections 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 --2- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f49e0038820 0x7f49e003ace0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 --2- 172.21.15.42:0/2431769073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49f4106ce0 0x7f49f407cad0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 >> 172.21.15.42:0/2431769073 conn(0x7f49f41022e0 msgr2=0x7f49f4102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 shutdown_connections 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:47.556+0000 7f49fbb31700 1 -- 172.21.15.42:0/2431769073 wait complete. 2023-12-12T09:49:49.015 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling cephadm module... 2023-12-12T09:49:50.361 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:50 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2357733774' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-12T09:49:50.361 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: ignoring --setuser ceph since I am not root 2023-12-12T09:49:50.361 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: ignoring --setgroup ceph since I am not root 2023-12-12T09:49:50.361 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:50.263+0000 7fb1e0833200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:49:50.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.536+0000 7f0e5e9d2700 1 Processor -- start 2023-12-12T09:49:50.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.536+0000 7f0e5e9d2700 1 -- start start 2023-12-12T09:49:50.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.536+0000 7f0e5e9d2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58109b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:50.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.537+0000 7f0e5e9d2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5807ae60 con 0x7f0e58107780 2023-12-12T09:49:50.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.537+0000 7f0e57fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58109b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:50.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.537+0000 7f0e57fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58109b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60364/0 (socket says 172.21.15.42:60364) 2023-12-12T09:49:50.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.537+0000 7f0e57fff700 1 -- 172.21.15.42:0/2599113909 learned_addr learned my addr 172.21.15.42:0/2599113909 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:50.467 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.537+0000 7f0e57fff700 1 -- 172.21.15.42:0/2599113909 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e5810a0b0 con 0x7f0e58107780 2023-12-12T09:49:50.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.537+0000 7f0e57fff700 1 --2- 172.21.15.42:0/2599113909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58109b70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0e480098a0 tx=0x7f0e48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f7a73aaa2ca0f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:50.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.537+0000 7f0e56ffd700 1 -- 172.21.15.42:0/2599113909 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e48004160 con 0x7f0e58107780 2023-12-12T09:49:50.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.537+0000 7f0e56ffd700 1 -- 172.21.15.42:0/2599113909 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e480042c0 con 0x7f0e58107780 2023-12-12T09:49:50.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.537+0000 7f0e56ffd700 1 -- 172.21.15.42:0/2599113909 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e48011a00 con 0x7f0e58107780 2023-12-12T09:49:50.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.538+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2599113909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 msgr2=0x7f0e58109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:50.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.538+0000 7f0e5e9d2700 1 --2- 172.21.15.42:0/2599113909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58109b70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0e480098a0 tx=0x7f0e48009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:50.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.538+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2599113909 shutdown_connections 2023-12-12T09:49:50.468 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.538+0000 7f0e5e9d2700 1 --2- 172.21.15.42:0/2599113909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58109b70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.538+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2599113909 >> 172.21.15.42:0/2599113909 conn(0x7f0e581011d0 msgr2=0x7f0e581035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.538+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2599113909 shutdown_connections 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.538+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2599113909 wait complete. 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.539+0000 7f0e5e9d2700 1 Processor -- start 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.539+0000 7f0e5e9d2700 1 -- start start 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.539+0000 7f0e5e9d2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58107190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.539+0000 7f0e5e9d2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58105730 con 0x7f0e58107780 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.539+0000 7f0e57fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58107190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.539+0000 7f0e57fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58107190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60366/0 (socket says 172.21.15.42:60366) 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.539+0000 7f0e57fff700 1 -- 172.21.15.42:0/2357733774 learned_addr learned my addr 172.21.15.42:0/2357733774 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.539+0000 7f0e57fff700 1 -- 172.21.15.42:0/2357733774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e48009550 con 0x7f0e58107780 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.540+0000 7f0e57fff700 1 --2- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58107190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009130 tx=0x7f0e48011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.540+0000 7f0e557fa700 1 -- 172.21.15.42:0/2357733774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e48003b70 con 0x7f0e58107780 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.540+0000 7f0e557fa700 1 -- 172.21.15.42:0/2357733774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0e48003cd0 con 0x7f0e58107780 2023-12-12T09:49:50.469 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.540+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e58105930 con 0x7f0e58107780 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.540+0000 7f0e557fa700 1 -- 172.21.15.42:0/2357733774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e480197d0 con 0x7f0e58107780 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.540+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e58105dd0 con 0x7f0e58107780 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.541+0000 7f0e557fa700 1 -- 172.21.15.42:0/2357733774 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) v1 ==== 45569+0+0 (secure 0 0 0) 0x7f0e480199c0 con 0x7f0e58107780 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.541+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e58067940 con 0x7f0e58107780 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.541+0000 7f0e557fa700 1 --2- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f0e40038820 0x7f0e4003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.541+0000 7f0e557fa700 1 -- 172.21.15.42:0/2357733774 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0e4804c9e0 con 0x7f0e58107780 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.541+0000 7f0e577fe700 1 --2- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f0e40038820 0x7f0e4003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.541+0000 7f0e577fe700 1 --2- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f0e40038820 0x7f0e4003ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c006fd0 tx=0x7f0e4c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:49.546+0000 7f0e557fa700 1 -- 172.21.15.42:0/2357733774 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0e4800b900 con 0x7f0e58107780 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.021+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f0e5810aeb0 con 0x7f0e58107780 2023-12-12T09:49:50.470 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.072+0000 7f0e557fa700 1 -- 172.21.15.42:0/2357733774 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f0e48018350 con 0x7f0e58107780 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.074+0000 7f0e557fa700 1 -- 172.21.15.42:0/2357733774 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f0e4800ed40 con 0x7f0e58107780 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.075+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f0e40038820 msgr2=0x7f0e4003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.075+0000 7f0e5e9d2700 1 --2- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f0e40038820 0x7f0e4003ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c006fd0 tx=0x7f0e4c006e40 comp rx=0 tx=0).stop 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.075+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 msgr2=0x7f0e58107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.075+0000 7f0e5e9d2700 1 --2- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58107190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009130 tx=0x7f0e48011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.076+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 shutdown_connections 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.076+0000 7f0e5e9d2700 1 --2- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f0e40038820 0x7f0e4003ace0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.076+0000 7f0e5e9d2700 1 --2- 172.21.15.42:0/2357733774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e58107780 0x7f0e58107190 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.076+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 >> 172.21.15.42:0/2357733774 conn(0x7f0e581011d0 msgr2=0x7f0e58101ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.076+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 shutdown_connections 2023-12-12T09:49:50.471 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.076+0000 7f0e5e9d2700 1 -- 172.21.15.42:0/2357733774 wait complete. 2023-12-12T09:49:50.652 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:50.359+0000 7fb1e0833200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:49:50.653 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:50.473+0000 7fb1e0833200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:49:50.653 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:50.595+0000 7fb1e0833200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:49:51.078 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:50.808+0000 7fb1e0833200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:49:51.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:51 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2357733774' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-12T09:49:51.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:51 smithi042 ceph-mon[98894]: mgrmap e5: x(active, since 11s) 2023-12-12T09:49:51.403 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:51 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:51.290+0000 7fb1e0833200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:49:51.804 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:51 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:51.442+0000 7fb1e0833200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:49:51.804 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:51 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:51.555+0000 7fb1e0833200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:49:51.804 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:51 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:51.670+0000 7fb1e0833200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:49:51.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2023-12-12T09:49:51.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-12-12T09:49:51.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-12T09:49:51.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-12T09:49:51.825 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.946+0000 7f03d2f89700 1 Processor -- start 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.946+0000 7f03d2f89700 1 -- start start 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03d2f89700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc109c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03d2f89700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc07ad80 con 0x7f03cc107880 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03d0d25700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03d0d25700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60380/0 (socket says 172.21.15.42:60380) 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03d0d25700 1 -- 172.21.15.42:0/3857727978 learned_addr learned my addr 172.21.15.42:0/3857727978 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03d0d25700 1 -- 172.21.15.42:0/3857727978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc07aec0 con 0x7f03cc107880 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03d0d25700 1 --2- 172.21.15.42:0/3857727978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc109c70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03c00098a0 tx=0x7f03c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebd12dcc34efc7a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03cb7fe700 1 -- 172.21.15.42:0/3857727978 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03c0004160 con 0x7f03cc107880 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03cb7fe700 1 -- 172.21.15.42:0/3857727978 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03c00042c0 con 0x7f03cc107880 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.947+0000 7f03cb7fe700 1 -- 172.21.15.42:0/3857727978 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03c0011a00 con 0x7f03cc107880 2023-12-12T09:49:51.826 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.948+0000 7f03d2f89700 1 -- 172.21.15.42:0/3857727978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 msgr2=0x7f03cc109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:51.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.948+0000 7f03d2f89700 1 --2- 172.21.15.42:0/3857727978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc109c70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03c00098a0 tx=0x7f03c0009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:51.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.948+0000 7f03d2f89700 1 -- 172.21.15.42:0/3857727978 shutdown_connections 2023-12-12T09:49:51.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.948+0000 7f03d2f89700 1 --2- 172.21.15.42:0/3857727978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc109c70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:51.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.948+0000 7f03d2f89700 1 -- 172.21.15.42:0/3857727978 >> 172.21.15.42:0/3857727978 conn(0x7f03cc1011d0 msgr2=0x7f03cc103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:51.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.948+0000 7f03d2f89700 1 -- 172.21.15.42:0/3857727978 shutdown_connections 2023-12-12T09:49:51.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.948+0000 7f03d2f89700 1 -- 172.21.15.42:0/3857727978 wait complete. 2023-12-12T09:49:51.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.949+0000 7f03d2f89700 1 Processor -- start 2023-12-12T09:49:51.827 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.949+0000 7f03d2f89700 1 -- start start 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.949+0000 7f03d2f89700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.949+0000 7f03d2f89700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc19af20 con 0x7f03cc107880 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.949+0000 7f03d0d25700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.949+0000 7f03d0d25700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60382/0 (socket says 172.21.15.42:60382) 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.949+0000 7f03d0d25700 1 -- 172.21.15.42:0/1578535267 learned_addr learned my addr 172.21.15.42:0/1578535267 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.950+0000 7f03d0d25700 1 -- 172.21.15.42:0/1578535267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0009550 con 0x7f03cc107880 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.950+0000 7f03d0d25700 1 --2- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc19a9e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009130 tx=0x7f03c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.950+0000 7f03c9ffb700 1 -- 172.21.15.42:0/1578535267 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03c0003b70 con 0x7f03cc107880 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.950+0000 7f03c9ffb700 1 -- 172.21.15.42:0/1578535267 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03c0003cd0 con 0x7f03cc107880 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.950+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc19b120 con 0x7f03cc107880 2023-12-12T09:49:51.828 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.950+0000 7f03c9ffb700 1 -- 172.21.15.42:0/1578535267 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03c00197d0 con 0x7f03cc107880 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.950+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc19b5c0 con 0x7f03cc107880 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.951+0000 7f03c9ffb700 1 -- 172.21.15.42:0/1578535267 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f03c0019a10 con 0x7f03cc107880 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.951+0000 7f03c9ffb700 1 --2- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f03bc038820 0x7f03bc03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.951+0000 7f03c9ffb700 1 -- 172.21.15.42:0/1578535267 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f03c004d580 con 0x7f03cc107880 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.951+0000 7f03cbfff700 1 -- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f03bc038820 msgr2=0x7f03bc03ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1433305616 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.951+0000 7f03cbfff700 1 --2- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f03bc038820 0x7f03bc03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.952+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc068600 con 0x7f03cc107880 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:50.957+0000 7f03c9ffb700 1 -- 172.21.15.42:0/1578535267 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f03c0018db0 con 0x7f03cc107880 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.152+0000 7f03cbfff700 1 -- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f03bc038820 msgr2=0x7f03bc03ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1433305616 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.152+0000 7f03cbfff700 1 --2- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f03bc038820 0x7f03bc03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.438+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f03cc10b120 con 0x7f03cc107880 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.439+0000 7f03c9ffb700 1 -- 172.21.15.42:0/1578535267 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f03c0018350 con 0x7f03cc107880 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.441+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f03bc038820 msgr2=0x7f03bc03ace0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:49:51.829 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.441+0000 7f03d2f89700 1 --2- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f03bc038820 0x7f03bc03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.441+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 msgr2=0x7f03cc19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.441+0000 7f03d2f89700 1 --2- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc19a9e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009130 tx=0x7f03c0011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.441+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 shutdown_connections 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.441+0000 7f03d2f89700 1 --2- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f03bc038820 0x7f03bc03ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.441+0000 7f03d2f89700 1 --2- 172.21.15.42:0/1578535267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03cc107880 0x7f03cc19a9e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.441+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 >> 172.21.15.42:0/1578535267 conn(0x7f03cc1011d0 msgr2=0x7f03cc101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.442+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 shutdown_connections 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:51.442+0000 7f03d2f89700 1 -- 172.21.15.42:0/1578535267 wait complete. 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2023-12-12T09:49:51.830 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 5... 2023-12-12T09:49:52.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:52 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1578535267' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-12T09:49:52.403 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:52 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:52.190+0000 7fb1e0833200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:49:53.402 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:52 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:52.977+0000 7fb1e0833200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:49:53.402 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:53 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:53.075+0000 7fb1e0833200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:49:54.382 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:54.047+0000 7fb1e0833200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:49:54.382 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:54.157+0000 7fb1e0833200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:49:55.152 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:54.380+0000 7fb1e0833200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:49:55.153 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:54.900+0000 7fb1e0833200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:49:55.153 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:55.007+0000 7fb1e0833200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:49:55.522 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:55.191+0000 7fb1e0833200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:49:55.902 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:55.520+0000 7fb1e0833200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:49:56.652 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:56.341+0000 7fb1e0833200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:49:56.653 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:56.499+0000 7fb1e0833200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:49:57.152 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:56.745+0000 7fb1e0833200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:49:57.152 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:56.880+0000 7fb1e0833200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:49:57.652 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:57 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:57.154+0000 7fb1e0833200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:49:58.652 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:49:58 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:49:58.191+0000 7fb1e0833200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:49:59.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: Active manager daemon x restarted 2023-12-12T09:49:59.607 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: Activating manager daemon x 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: osdmap e2: 0 total, 0 up, 0 in 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: mgrmap e6: x(active, starting, since 0.00343108s) 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: Manager daemon x is now available 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: Found migration_current of "None". Setting to last migration. 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:49:59.608 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:49:59.609 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-12T09:49:59.609 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-12T09:49:59.609 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:49:59.609 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:49:59 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:49:59.688 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2023-12-12T09:49:59.688 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-12-12T09:49:59.688 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.315+0000 7f69f8a3f700 1 Processor -- start 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.315+0000 7f69f8a3f700 1 -- start start 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.315+0000 7f69f8a3f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f4107100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.315+0000 7f69f8a3f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f41076d0 con 0x7f69f4106ce0 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.316+0000 7f69f259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f4107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.316+0000 7f69f259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f4107100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60390/0 (socket says 172.21.15.42:60390) 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.316+0000 7f69f259c700 1 -- 172.21.15.42:0/1232834466 learned_addr learned my addr 172.21.15.42:0/1232834466 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.316+0000 7f69f259c700 1 -- 172.21.15.42:0/1232834466 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f4107ee0 con 0x7f69f4106ce0 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.316+0000 7f69f259c700 1 --2- 172.21.15.42:0/1232834466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f4107100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f69e40098a0 tx=0x7f69e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd5d9e1ba516899 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.316+0000 7f69f159a700 1 -- 172.21.15.42:0/1232834466 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69e4004160 con 0x7f69f4106ce0 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.316+0000 7f69f159a700 1 -- 172.21.15.42:0/1232834466 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69e40042c0 con 0x7f69f4106ce0 2023-12-12T09:49:59.689 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.316+0000 7f69f159a700 1 -- 172.21.15.42:0/1232834466 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69e4011a00 con 0x7f69f4106ce0 2023-12-12T09:49:59.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.317+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1232834466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 msgr2=0x7f69f4107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:59.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.317+0000 7f69f8a3f700 1 --2- 172.21.15.42:0/1232834466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f4107100 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f69e40098a0 tx=0x7f69e4009bb0 comp rx=0 tx=0).stop 2023-12-12T09:49:59.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.317+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1232834466 shutdown_connections 2023-12-12T09:49:59.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.317+0000 7f69f8a3f700 1 --2- 172.21.15.42:0/1232834466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f4107100 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:59.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.317+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1232834466 >> 172.21.15.42:0/1232834466 conn(0x7f69f41022e0 msgr2=0x7f69f4104700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:59.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.317+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1232834466 shutdown_connections 2023-12-12T09:49:59.690 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.317+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1232834466 wait complete. 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.318+0000 7f69f8a3f700 1 Processor -- start 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.318+0000 7f69f8a3f700 1 -- start start 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.318+0000 7f69f8a3f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f407cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.318+0000 7f69f8a3f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f407d010 con 0x7f69f4106ce0 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.318+0000 7f69f259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f407cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.318+0000 7f69f259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f407cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60392/0 (socket says 172.21.15.42:60392) 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.318+0000 7f69f259c700 1 -- 172.21.15.42:0/1252870305 learned_addr learned my addr 172.21.15.42:0/1252870305 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.319+0000 7f69f259c700 1 -- 172.21.15.42:0/1252870305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e4009550 con 0x7f69f4106ce0 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.319+0000 7f69f259c700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f407cad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009130 tx=0x7f69e4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.319+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69e4003ab0 con 0x7f69f4106ce0 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.319+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1252870305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f407b250 con 0x7f69f4106ce0 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.319+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f69e4003c10 con 0x7f69f4106ce0 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.319+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69e40247d0 con 0x7f69f4106ce0 2023-12-12T09:49:59.691 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.319+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1252870305 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f407b6f0 con 0x7f69f4106ce0 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.320+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f69e402d3f0 con 0x7f69f4106ce0 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.320+0000 7f69e37fe700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 0x7f69dc03ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.320+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 --> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69dc03b3f0 con 0x7f69dc038820 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.320+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f69e404c610 con 0x7f69f4106ce0 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.320+0000 7f69f1d9b700 1 -- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 msgr2=0x7f69dc03ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1433305616 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.320+0000 7f69f1d9b700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 0x7f69dc03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.521+0000 7f69f1d9b700 1 -- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 msgr2=0x7f69dc03ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1433305616 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.521+0000 7f69f1d9b700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 0x7f69dc03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.922+0000 7f69f1d9b700 1 -- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 msgr2=0x7f69dc03ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1433305616 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:52.922+0000 7f69f1d9b700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 0x7f69dc03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:53.723+0000 7f69f1d9b700 1 -- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 msgr2=0x7f69dc03ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1433305616 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:53.723+0000 7f69f1d9b700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 0x7f69dc03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:55.324+0000 7f69f1d9b700 1 -- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 msgr2=0x7f69dc03ace0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1433305616 2023-12-12T09:49:59.692 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:55.324+0000 7f69f1d9b700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 0x7f69dc03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:58.207+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 6) v1 ==== 45347+0+0 (secure 0 0 0) 0x7f69e4019460 con 0x7f69f4106ce0 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:58.207+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 msgr2=0x7f69dc03ace0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:58.207+0000 7f69e37fe700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 0x7f69dc03ace0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.208+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 7) v1 ==== 45474+0+0 (secure 0 0 0) 0x7f69e404de60 con 0x7f69f4106ce0 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.208+0000 7f69e37fe700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f69dc03bff0 0x7f69dc03e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.208+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69dc03b3f0 con 0x7f69dc03bff0 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.209+0000 7f69f1d9b700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f69dc03bff0 0x7f69dc03e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.209+0000 7f69f1d9b700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f69dc03bff0 0x7f69dc03e3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f69e8003a10 tx=0x7f69e8009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.210+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 <== mgr.14120 v2:172.21.15.42:6800/1525180467 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f69dc03b3f0 con 0x7f69dc03bff0 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1252870305 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f69f4108d50 con 0x7f69dc03bff0 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69e37fe700 1 -- 172.21.15.42:0/1252870305 <== mgr.14120 v2:172.21.15.42:6800/1525180467 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f69e8008850 con 0x7f69dc03bff0 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f69dc03bff0 msgr2=0x7f69dc03e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f69dc03bff0 0x7f69dc03e3e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f69e8003a10 tx=0x7f69e8009290 comp rx=0 tx=0).stop 2023-12-12T09:49:59.693 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 msgr2=0x7f69f407cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f407cad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009130 tx=0x7f69e4011ad0 comp rx=0 tx=0).stop 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1252870305 shutdown_connections 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f69dc03bff0 0x7f69dc03e3e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69f4106ce0 0x7f69f407cad0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 --2- 172.21.15.42:0/1252870305 >> [v2:172.21.15.42:6800/1433305616,v1:172.21.15.42:6801/1433305616] conn(0x7f69dc038820 0x7f69dc03ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1252870305 >> 172.21.15.42:0/1252870305 conn(0x7f69f41022e0 msgr2=0x7f69f4102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1252870305 shutdown_connections 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:49:59.225+0000 7f69f8a3f700 1 -- 172.21.15.42:0/1252870305 wait complete. 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 5 is available 2023-12-12T09:49:59.694 INFO:teuthology.orchestra.run.smithi042.stdout:Setting orchestrator backend to cephadm... 2023-12-12T09:50:00.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:00 smithi042 ceph-mon[98894]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-12T09:50:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:00 smithi042 ceph-mon[98894]: mgrmap e7: x(active, since 1.00506s) 2023-12-12T09:50:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:00 smithi042 ceph-mon[98894]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-12T09:50:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:00 smithi042 ceph-mon[98894]: [12/Dec/2023:09:49:59] ENGINE Bus STARTING 2023-12-12T09:50:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:00 smithi042 ceph-mon[98894]: [12/Dec/2023:09:49:59] ENGINE Serving on https://172.21.15.42:7150 2023-12-12T09:50:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:00 smithi042 ceph-mon[98894]: [12/Dec/2023:09:49:59] ENGINE Serving on http://172.21.15.42:8765 2023-12-12T09:50:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:00 smithi042 ceph-mon[98894]: [12/Dec/2023:09:49:59] ENGINE Bus STARTED 2023-12-12T09:50:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:00 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.222+0000 7f4be56e4700 1 Processor -- start 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.222+0000 7f4be56e4700 1 -- start start 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.222+0000 7f4be56e4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be0106020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.222+0000 7f4be56e4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be01065f0 con 0x7f4be0105c00 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.222+0000 7f4bdeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be0106020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.223+0000 7f4bdeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be0106020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60444/0 (socket says 172.21.15.42:60444) 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.223+0000 7f4bdeffd700 1 -- 172.21.15.42:0/722345867 learned_addr learned my addr 172.21.15.42:0/722345867 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.223+0000 7f4bdeffd700 1 -- 172.21.15.42:0/722345867 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be0106e00 con 0x7f4be0105c00 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.223+0000 7f4bdeffd700 1 --2- 172.21.15.42:0/722345867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be0106020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00098a0 tx=0x7f4bd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d900801e7bb81365 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:01.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.223+0000 7f4bddffb700 1 -- 172.21.15.42:0/722345867 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bd0004160 con 0x7f4be0105c00 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.223+0000 7f4bddffb700 1 -- 172.21.15.42:0/722345867 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bd00042c0 con 0x7f4be0105c00 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.223+0000 7f4bddffb700 1 -- 172.21.15.42:0/722345867 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bd0011a00 con 0x7f4be0105c00 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.224+0000 7f4be56e4700 1 -- 172.21.15.42:0/722345867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 msgr2=0x7f4be0106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.224+0000 7f4be56e4700 1 --2- 172.21.15.42:0/722345867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be0106020 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4bd00098a0 tx=0x7f4bd0009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.224+0000 7f4be56e4700 1 -- 172.21.15.42:0/722345867 shutdown_connections 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.224+0000 7f4be56e4700 1 --2- 172.21.15.42:0/722345867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be0106020 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.224+0000 7f4be56e4700 1 -- 172.21.15.42:0/722345867 >> 172.21.15.42:0/722345867 conn(0x7f4be0101200 msgr2=0x7f4be0103620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.224+0000 7f4be56e4700 1 -- 172.21.15.42:0/722345867 shutdown_connections 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.224+0000 7f4be56e4700 1 -- 172.21.15.42:0/722345867 wait complete. 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.224+0000 7f4be56e4700 1 Processor -- start 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.225+0000 7f4be56e4700 1 -- start start 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.225+0000 7f4be56e4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.225+0000 7f4be56e4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be007ab90 con 0x7f4be0105c00 2023-12-12T09:50:01.034 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.225+0000 7f4bdeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.225+0000 7f4bdeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be019d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60446/0 (socket says 172.21.15.42:60446) 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.225+0000 7f4bdeffd700 1 -- 172.21.15.42:0/3629281379 learned_addr learned my addr 172.21.15.42:0/3629281379 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.225+0000 7f4bdeffd700 1 -- 172.21.15.42:0/3629281379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0009550 con 0x7f4be0105c00 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.225+0000 7f4bdeffd700 1 --2- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be019d780 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009fb0 tx=0x7f4bd0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.225+0000 7f4bd7fff700 1 -- 172.21.15.42:0/3629281379 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bd0003a40 con 0x7f4be0105c00 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.226+0000 7f4bd7fff700 1 -- 172.21.15.42:0/3629281379 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4bd0003ba0 con 0x7f4be0105c00 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.226+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be007ad90 con 0x7f4be0105c00 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.226+0000 7f4bd7fff700 1 -- 172.21.15.42:0/3629281379 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4bd001a750 con 0x7f4be0105c00 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.226+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be019de90 con 0x7f4be0105c00 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.227+0000 7f4bd7fff700 1 -- 172.21.15.42:0/3629281379 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 7) v1 ==== 45474+0+0 (secure 0 0 0) 0x7f4bd0024430 con 0x7f4be0105c00 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.227+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be0067940 con 0x7f4be0105c00 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.227+0000 7f4bd7fff700 1 --2- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4bcc0386b0 0x7f4bcc03ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.227+0000 7f4bd7fff700 1 -- 172.21.15.42:0/3629281379 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4bd004c6f0 con 0x7f4be0105c00 2023-12-12T09:50:01.035 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.227+0000 7f4bde7fc700 1 --2- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4bcc0386b0 0x7f4bcc03ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.227+0000 7f4bde7fc700 1 --2- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4bcc0386b0 0x7f4bcc03ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8006fd0 tx=0x7f4bc8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.232+0000 7f4bd7fff700 1 -- 172.21.15.42:0/3629281379 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4bd00189a0 con 0x7f4be0105c00 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.624+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f4be0102450 con 0x7f4bcc0386b0 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.630+0000 7f4bd7fff700 1 -- 172.21.15.42:0/3629281379 <== mgr.14120 v2:172.21.15.42:6800/1525180467 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4be0102450 con 0x7f4bcc0386b0 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4bcc0386b0 msgr2=0x7f4bcc03ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 --2- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4bcc0386b0 0x7f4bcc03ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8006fd0 tx=0x7f4bc8006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 msgr2=0x7f4be019d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 --2- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be019d780 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0009fb0 tx=0x7f4bd0011ad0 comp rx=0 tx=0).stop 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 shutdown_connections 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 --2- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4bcc0386b0 0x7f4bcc03ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 --2- 172.21.15.42:0/3629281379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4be0105c00 0x7f4be019d780 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 >> 172.21.15.42:0/3629281379 conn(0x7f4be0101200 msgr2=0x7f4be0101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 shutdown_connections 2023-12-12T09:50:01.036 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:00.633+0000 7f4be56e4700 1 -- 172.21.15.42:0/3629281379 wait complete. 2023-12-12T09:50:01.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:01 smithi042 ceph-mon[98894]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:01.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:01 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:01.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:01 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:50:01.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:01 smithi042 ceph-mon[98894]: mgrmap e8: x(active, since 3s) 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout value unchanged 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.545+0000 7f424206e700 1 Processor -- start 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.545+0000 7f424206e700 1 -- start start 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.545+0000 7f424206e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c108770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.545+0000 7f424206e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c108d40 con 0x7f423c108350 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.545+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c108770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.545+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c108770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60450/0 (socket says 172.21.15.42:60450) 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.545+0000 7f423b7fe700 1 -- 172.21.15.42:0/1606288685 learned_addr learned my addr 172.21.15.42:0/1606288685 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.546+0000 7f423b7fe700 1 -- 172.21.15.42:0/1606288685 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c108e80 con 0x7f423c108350 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.546+0000 7f423b7fe700 1 --2- 172.21.15.42:0/1606288685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c108770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f422c0098a0 tx=0x7f422c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abe631647bd9580 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.546+0000 7f423a7fc700 1 -- 172.21.15.42:0/1606288685 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f422c004160 con 0x7f423c108350 2023-12-12T09:50:02.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.546+0000 7f423a7fc700 1 -- 172.21.15.42:0/1606288685 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f422c0042c0 con 0x7f423c108350 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.546+0000 7f423a7fc700 1 -- 172.21.15.42:0/1606288685 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f422c011a00 con 0x7f423c108350 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.547+0000 7f424206e700 1 -- 172.21.15.42:0/1606288685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 msgr2=0x7f423c108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.547+0000 7f424206e700 1 --2- 172.21.15.42:0/1606288685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c108770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f422c0098a0 tx=0x7f422c009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.547+0000 7f424206e700 1 -- 172.21.15.42:0/1606288685 shutdown_connections 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.547+0000 7f424206e700 1 --2- 172.21.15.42:0/1606288685 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c108770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.547+0000 7f424206e700 1 -- 172.21.15.42:0/1606288685 >> 172.21.15.42:0/1606288685 conn(0x7f423c100950 msgr2=0x7f423c102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.547+0000 7f424206e700 1 -- 172.21.15.42:0/1606288685 shutdown_connections 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.547+0000 7f424206e700 1 -- 172.21.15.42:0/1606288685 wait complete. 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.547+0000 7f424206e700 1 Processor -- start 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.548+0000 7f424206e700 1 -- start start 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.548+0000 7f424206e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.548+0000 7f424206e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423c19af80 con 0x7f423c108350 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.548+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:02.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.548+0000 7f423b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60452/0 (socket says 172.21.15.42:60452) 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.548+0000 7f423b7fe700 1 -- 172.21.15.42:0/3890398592 learned_addr learned my addr 172.21.15.42:0/3890398592 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.548+0000 7f423b7fe700 1 -- 172.21.15.42:0/3890398592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f422c009550 con 0x7f423c108350 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.549+0000 7f423b7fe700 1 --2- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c19aa40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f422c009130 tx=0x7f422c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.549+0000 7f4238ff9700 1 -- 172.21.15.42:0/3890398592 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f422c003ab0 con 0x7f423c108350 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.549+0000 7f4238ff9700 1 -- 172.21.15.42:0/3890398592 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f422c003c10 con 0x7f423c108350 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.549+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f423c19b180 con 0x7f423c108350 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.549+0000 7f4238ff9700 1 -- 172.21.15.42:0/3890398592 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f422c0247d0 con 0x7f423c108350 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.549+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f423c19b620 con 0x7f423c108350 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.550+0000 7f4238ff9700 1 -- 172.21.15.42:0/3890398592 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f422c02d3f0 con 0x7f423c108350 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.550+0000 7f4238ff9700 1 --2- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4224038820 0x7f422403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.550+0000 7f4238ff9700 1 -- 172.21.15.42:0/3890398592 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f422c04cc30 con 0x7f423c108350 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.550+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f423c1059f0 con 0x7f423c108350 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.550+0000 7f423affd700 1 --2- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4224038820 0x7f422403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:02.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.550+0000 7f423affd700 1 --2- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4224038820 0x7f422403ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4230006fd0 tx=0x7f4230006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.555+0000 7f4238ff9700 1 -- 172.21.15.42:0/3890398592 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f422c010740 con 0x7f423c108350 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.926+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f423c101b70 con 0x7f4224038820 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.927+0000 7f4238ff9700 1 -- 172.21.15.42:0/3890398592 <== mgr.14120 v2:172.21.15.42:6800/1525180467 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f423c101b70 con 0x7f4224038820 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.929+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4224038820 msgr2=0x7f422403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.929+0000 7f424206e700 1 --2- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4224038820 0x7f422403ace0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4230006fd0 tx=0x7f4230006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.929+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 msgr2=0x7f423c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.929+0000 7f424206e700 1 --2- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c19aa40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f422c009130 tx=0x7f422c011ad0 comp rx=0 tx=0).stop 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.929+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 shutdown_connections 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.929+0000 7f424206e700 1 --2- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f4224038820 0x7f422403ace0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.929+0000 7f424206e700 1 --2- 172.21.15.42:0/3890398592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f423c108350 0x7f423c19aa40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.929+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 >> 172.21.15.42:0/3890398592 conn(0x7f423c100950 msgr2=0x7f423c101460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.930+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 shutdown_connections 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:01.930+0000 7f424206e700 1 -- 172.21.15.42:0/3890398592 wait complete. 2023-12-12T09:50:02.284 INFO:teuthology.orchestra.run.smithi042.stdout:Generating ssh key... 2023-12-12T09:50:02.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:02 smithi042 ceph-mon[98894]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:03.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: Generating public/private rsa key pair. 2023-12-12T09:50:03.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: Your identification has been saved in /tmp/tmpt8sskt6m/key. 2023-12-12T09:50:03.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: Your public key has been saved in /tmp/tmpt8sskt6m/key.pub. 2023-12-12T09:50:03.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: The key fingerprint is: 2023-12-12T09:50:03.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: SHA256:ieJ+fG8KyOhmmJMz2PW4ytNApAlPo2ziKeJPC9v2/2w ceph-968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:50:03.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: The key's randomart image is: 2023-12-12T09:50:03.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: +---[RSA 3072]----+ 2023-12-12T09:50:03.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: | | 2023-12-12T09:50:03.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: |. + | 2023-12-12T09:50:03.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: |oB . | 2023-12-12T09:50:03.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: |=oo . . | 2023-12-12T09:50:03.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: |+.. . . S | 2023-12-12T09:50:03.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: |oo.+.o | 2023-12-12T09:50:03.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: |=*.=++. | 2023-12-12T09:50:03.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: |O+Xo+ +.oE. | 2023-12-12T09:50:03.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: | B**++.o+*. | 2023-12-12T09:50:03.904 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: +----[SHA256]-----+ 2023-12-12T09:50:04.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.800+0000 7f5f362ce700 1 Processor -- start 2023-12-12T09:50:04.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.800+0000 7f5f362ce700 1 -- start start 2023-12-12T09:50:04.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.800+0000 7f5f362ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f30106550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:04.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.800+0000 7f5f362ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f30106b20 con 0x7f5f30106130 2023-12-12T09:50:04.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.800+0000 7f5f2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f30106550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:04.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.800+0000 7f5f2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f30106550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60456/0 (socket says 172.21.15.42:60456) 2023-12-12T09:50:04.029 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.800+0000 7f5f2ffff700 1 -- 172.21.15.42:0/1335031720 learned_addr learned my addr 172.21.15.42:0/1335031720 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:04.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.801+0000 7f5f2ffff700 1 -- 172.21.15.42:0/1335031720 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f30106e50 con 0x7f5f30106130 2023-12-12T09:50:04.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.801+0000 7f5f2ffff700 1 --2- 172.21.15.42:0/1335031720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f30106550 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5f200098a0 tx=0x7f5f20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c807090f6bb1bcd server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:04.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.801+0000 7f5f2effd700 1 -- 172.21.15.42:0/1335031720 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f20004160 con 0x7f5f30106130 2023-12-12T09:50:04.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.801+0000 7f5f2effd700 1 -- 172.21.15.42:0/1335031720 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f200042c0 con 0x7f5f30106130 2023-12-12T09:50:04.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.801+0000 7f5f2effd700 1 -- 172.21.15.42:0/1335031720 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f20011a00 con 0x7f5f30106130 2023-12-12T09:50:04.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.802+0000 7f5f362ce700 1 -- 172.21.15.42:0/1335031720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 msgr2=0x7f5f30106550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:04.030 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.802+0000 7f5f362ce700 1 --2- 172.21.15.42:0/1335031720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f30106550 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5f200098a0 tx=0x7f5f20009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.802+0000 7f5f362ce700 1 -- 172.21.15.42:0/1335031720 shutdown_connections 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.802+0000 7f5f362ce700 1 --2- 172.21.15.42:0/1335031720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f30106550 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.802+0000 7f5f362ce700 1 -- 172.21.15.42:0/1335031720 >> 172.21.15.42:0/1335031720 conn(0x7f5f30100950 msgr2=0x7f5f30102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.802+0000 7f5f362ce700 1 -- 172.21.15.42:0/1335031720 shutdown_connections 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.802+0000 7f5f362ce700 1 -- 172.21.15.42:0/1335031720 wait complete. 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.802+0000 7f5f362ce700 1 Processor -- start 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.803+0000 7f5f362ce700 1 -- start start 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.803+0000 7f5f362ce700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f3019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.803+0000 7f5f362ce700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3019d130 con 0x7f5f30106130 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.803+0000 7f5f2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f3019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.803+0000 7f5f2ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f3019cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60458/0 (socket says 172.21.15.42:60458) 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.803+0000 7f5f2ffff700 1 -- 172.21.15.42:0/2953082424 learned_addr learned my addr 172.21.15.42:0/2953082424 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.803+0000 7f5f2ffff700 1 -- 172.21.15.42:0/2953082424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20009550 con 0x7f5f30106130 2023-12-12T09:50:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.803+0000 7f5f2ffff700 1 --2- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f3019cbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009130 tx=0x7f5f20011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.804+0000 7f5f2d7fa700 1 -- 172.21.15.42:0/2953082424 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f20003b70 con 0x7f5f30106130 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.804+0000 7f5f2d7fa700 1 -- 172.21.15.42:0/2953082424 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f20003cd0 con 0x7f5f30106130 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.804+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3019d330 con 0x7f5f30106130 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.804+0000 7f5f2d7fa700 1 -- 172.21.15.42:0/2953082424 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f200197d0 con 0x7f5f30106130 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.804+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3019d7d0 con 0x7f5f30106130 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.805+0000 7f5f2d7fa700 1 -- 172.21.15.42:0/2953082424 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f5f200199c0 con 0x7f5f30106130 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.805+0000 7f5f2d7fa700 1 --2- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f5f18038870 0x7f5f1803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.805+0000 7f5f2d7fa700 1 -- 172.21.15.42:0/2953082424 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5f2004ca10 con 0x7f5f30106130 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.805+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f1c005320 con 0x7f5f30106130 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.805+0000 7f5f2f7fe700 1 --2- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f5f18038870 0x7f5f1803ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.805+0000 7f5f2f7fe700 1 --2- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f5f18038870 0x7f5f1803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006fd0 tx=0x7f5f24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:02.810+0000 7f5f2d7fa700 1 -- 172.21.15.42:0/2953082424 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5f20018db0 con 0x7f5f30106130 2023-12-12T09:50:04.032 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.180+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f1c000bf0 con 0x7f5f18038870 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.627+0000 7f5f2d7fa700 1 -- 172.21.15.42:0/2953082424 <== mgr.14120 v2:172.21.15.42:6800/1525180467 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5f1c000bf0 con 0x7f5f18038870 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f5f18038870 msgr2=0x7f5f1803ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 --2- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f5f18038870 0x7f5f1803ad30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5f24006fd0 tx=0x7f5f24006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 msgr2=0x7f5f3019cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 --2- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f3019cbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f20009130 tx=0x7f5f20011ad0 comp rx=0 tx=0).stop 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 shutdown_connections 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 --2- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f5f18038870 0x7f5f1803ad30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 --2- 172.21.15.42:0/2953082424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f30106130 0x7f5f3019cbf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 >> 172.21.15.42:0/2953082424 conn(0x7f5f30100950 msgr2=0x7f5f30101810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 shutdown_connections 2023-12-12T09:50:04.033 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:03.630+0000 7f5f362ce700 1 -- 172.21.15.42:0/2953082424 wait complete. 2023-12-12T09:50:04.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:04 smithi042 ceph-mon[98894]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:04.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:04 smithi042 ceph-mon[98894]: Generating ssh key... 2023-12-12T09:50:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:04 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:04 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDt4fi1xpyNYb3S4s0HRhz63+Pa8ivzFs+59IgK+zylN60YKGI5JVKjA+b19YJrRahc7k+ejtWYV3ReCCd6LhQ7rvuSxVEKAMBLxfGXpGmT4NeiCb0T0hoi+D7MUedjEYFdqIudjtyC++4KC2Yt7BQ4yBsgajVWhalzT776devXwujy4oBVjF1ffVTt+JZFHCPhTfbaHEI/CLXgaCqVS7Owtg2A3jJOa65drkTTU98+AI0feBIvn2BkS5DQxdVcKI8EuZzIkOhrv2SfXFUDUEAlO0TrrEHpwZXexAb4SSixwKu+uBJaZ7iyawQFcb+p8jq+w0lFiqFiL+Qw4COl/FPSE+e8pYBvDx4GSjIuyHjtFZV4TVdQuFgMGDUk0EL1l704VmrGv/9x3wxUYm71/Jhc66uXPsJmU/XRiP4QaYhFb1ETKKReR8TIDiogjk56CmlMSoPHNchMoiic+w5MOa6Zp5NtRP6RV9J5QXDXNJqlnO1Bz4XADhryCj36mMPmiPs= ceph-968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:50:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.534+0000 7faeeee90700 1 Processor -- start 2023-12-12T09:50:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.534+0000 7faeeee90700 1 -- start start 2023-12-12T09:50:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.534+0000 7faeeee90700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee8105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.534+0000 7faeeee90700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee8105a10 con 0x7faee8105020 2023-12-12T09:50:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.534+0000 7faeecc2c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee8105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.534+0000 7faeecc2c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee8105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60462/0 (socket says 172.21.15.42:60462) 2023-12-12T09:50:05.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.534+0000 7faeecc2c700 1 -- 172.21.15.42:0/1627342924 learned_addr learned my addr 172.21.15.42:0/1627342924 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.534+0000 7faeecc2c700 1 -- 172.21.15.42:0/1627342924 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8107fb0 con 0x7faee8105020 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.535+0000 7faeecc2c700 1 --2- 172.21.15.42:0/1627342924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee8105440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faedc0098a0 tx=0x7faedc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61f1011ea3692cb server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.535+0000 7faee77fe700 1 -- 172.21.15.42:0/1627342924 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faedc004160 con 0x7faee8105020 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.535+0000 7faee77fe700 1 -- 172.21.15.42:0/1627342924 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faedc0042c0 con 0x7faee8105020 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.535+0000 7faee77fe700 1 -- 172.21.15.42:0/1627342924 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faedc011a00 con 0x7faee8105020 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.535+0000 7faeeee90700 1 -- 172.21.15.42:0/1627342924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 msgr2=0x7faee8105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.535+0000 7faeeee90700 1 --2- 172.21.15.42:0/1627342924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee8105440 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faedc0098a0 tx=0x7faedc009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.535+0000 7faeeee90700 1 -- 172.21.15.42:0/1627342924 shutdown_connections 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.535+0000 7faeeee90700 1 --2- 172.21.15.42:0/1627342924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee8105440 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.535+0000 7faeeee90700 1 -- 172.21.15.42:0/1627342924 >> 172.21.15.42:0/1627342924 conn(0x7faee8100950 msgr2=0x7faee8102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.536+0000 7faeeee90700 1 -- 172.21.15.42:0/1627342924 shutdown_connections 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.536+0000 7faeeee90700 1 -- 172.21.15.42:0/1627342924 wait complete. 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.536+0000 7faeeee90700 1 Processor -- start 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.536+0000 7faeeee90700 1 -- start start 2023-12-12T09:50:05.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.536+0000 7faeeee90700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee81077e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.536+0000 7faeeee90700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee8107d20 con 0x7faee8105020 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.536+0000 7faeecc2c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee81077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.536+0000 7faeecc2c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee81077e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60464/0 (socket says 172.21.15.42:60464) 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.536+0000 7faeecc2c700 1 -- 172.21.15.42:0/3983097379 learned_addr learned my addr 172.21.15.42:0/3983097379 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.537+0000 7faeecc2c700 1 -- 172.21.15.42:0/3983097379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc009550 con 0x7faee8105020 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.537+0000 7faeecc2c700 1 --2- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee81077e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faedc009fb0 tx=0x7faedc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.537+0000 7faee5ffb700 1 -- 172.21.15.42:0/3983097379 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faedc003a40 con 0x7faee8105020 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.537+0000 7faee5ffb700 1 -- 172.21.15.42:0/3983097379 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7faedc003ba0 con 0x7faee8105020 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.537+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee8105f60 con 0x7faee8105020 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.537+0000 7faee5ffb700 1 -- 172.21.15.42:0/3983097379 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faedc01a750 con 0x7faee8105020 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.537+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee8106400 con 0x7faee8105020 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.538+0000 7faee5ffb700 1 -- 172.21.15.42:0/3983097379 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7faedc024b60 con 0x7faee8105020 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.538+0000 7faee5ffb700 1 --2- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7faed80387d0 0x7faed803ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:05.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.538+0000 7faee5ffb700 1 -- 172.21.15.42:0/3983097379 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faedc04c810 con 0x7faee8105020 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.538+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee8068600 con 0x7faee8105020 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.539+0000 7faee7fff700 1 --2- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7faed80387d0 0x7faed803ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.539+0000 7faee7fff700 1 --2- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7faed80387d0 0x7faed803ac90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faed4006fd0 tx=0x7faed4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.543+0000 7faee5ffb700 1 -- 172.21.15.42:0/3983097379 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faedc02d3f0 con 0x7faee8105020 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.909+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7faee8106b60 con 0x7faed80387d0 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.910+0000 7faee5ffb700 1 -- 172.21.15.42:0/3983097379 <== mgr.14120 v2:172.21.15.42:6800/1525180467 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7faee8106b60 con 0x7faed80387d0 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.912+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7faed80387d0 msgr2=0x7faed803ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.912+0000 7faeeee90700 1 --2- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7faed80387d0 0x7faed803ac90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7faed4006fd0 tx=0x7faed4006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.912+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 msgr2=0x7faee81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.912+0000 7faeeee90700 1 --2- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee81077e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faedc009fb0 tx=0x7faedc011ad0 comp rx=0 tx=0).stop 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.912+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 shutdown_connections 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.912+0000 7faeeee90700 1 --2- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7faed80387d0 0x7faed803ac90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:05.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.912+0000 7faeeee90700 1 --2- 172.21.15.42:0/3983097379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faee8105020 0x7faee81077e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.912+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 >> 172.21.15.42:0/3983097379 conn(0x7faee8100950 msgr2=0x7faee8101510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.913+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 shutdown_connections 2023-12-12T09:50:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:04.913+0000 7faeeee90700 1 -- 172.21.15.42:0/3983097379 wait complete. 2023-12-12T09:50:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-12T09:50:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:Adding key to root@localhost authorized_keys... 2023-12-12T09:50:05.280 INFO:teuthology.orchestra.run.smithi042.stdout:Adding host smithi042... 2023-12-12T09:50:05.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:05 smithi042 ceph-mon[98894]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:07.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:07 smithi042 ceph-mon[98894]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:07.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:07 smithi042 ceph-mon[98894]: Deploying cephadm binary to smithi042 2023-12-12T09:50:12.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Added host 'smithi042' with addr '172.21.15.42' 2023-12-12T09:50:12.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.830+0000 7ff8c6efc700 1 Processor -- start 2023-12-12T09:50:12.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.830+0000 7ff8c6efc700 1 -- start start 2023-12-12T09:50:12.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.830+0000 7ff8c6efc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c007b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:12.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.830+0000 7ff8c6efc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c007b8a0 con 0x7ff8c007ce70 2023-12-12T09:50:12.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.830+0000 7ff8c4c98700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c007b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:12.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.830+0000 7ff8c4c98700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c007b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60468/0 (socket says 172.21.15.42:60468) 2023-12-12T09:50:12.037 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.830+0000 7ff8c4c98700 1 -- 172.21.15.42:0/2939596592 learned_addr learned my addr 172.21.15.42:0/2939596592 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.830+0000 7ff8c4c98700 1 -- 172.21.15.42:0/2939596592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c007b9e0 con 0x7ff8c007ce70 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.831+0000 7ff8c4c98700 1 --2- 172.21.15.42:0/2939596592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c007b2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40098a0 tx=0x7ff8b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dea985811c76e8d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.831+0000 7ff8bf7fe700 1 -- 172.21.15.42:0/2939596592 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4004160 con 0x7ff8c007ce70 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.831+0000 7ff8bf7fe700 1 -- 172.21.15.42:0/2939596592 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8b40042c0 con 0x7ff8c007ce70 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.831+0000 7ff8bf7fe700 1 -- 172.21.15.42:0/2939596592 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4011a00 con 0x7ff8c007ce70 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.831+0000 7ff8c6efc700 1 -- 172.21.15.42:0/2939596592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 msgr2=0x7ff8c007b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.831+0000 7ff8c6efc700 1 --2- 172.21.15.42:0/2939596592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c007b2d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff8b40098a0 tx=0x7ff8b4009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.831+0000 7ff8c6efc700 1 -- 172.21.15.42:0/2939596592 shutdown_connections 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.831+0000 7ff8c6efc700 1 --2- 172.21.15.42:0/2939596592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c007b2d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.831+0000 7ff8c6efc700 1 -- 172.21.15.42:0/2939596592 >> 172.21.15.42:0/2939596592 conn(0x7ff8c01022d0 msgr2=0x7ff8c01046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.832+0000 7ff8c6efc700 1 -- 172.21.15.42:0/2939596592 shutdown_connections 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.832+0000 7ff8c6efc700 1 -- 172.21.15.42:0/2939596592 wait complete. 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.832+0000 7ff8c6efc700 1 Processor -- start 2023-12-12T09:50:12.038 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.832+0000 7ff8c6efc700 1 -- start start 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.832+0000 7ff8c6efc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c019ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.832+0000 7ff8c6efc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c019d230 con 0x7ff8c007ce70 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.832+0000 7ff8c4c98700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c019ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.832+0000 7ff8c4c98700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c019ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60470/0 (socket says 172.21.15.42:60470) 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.832+0000 7ff8c4c98700 1 -- 172.21.15.42:0/4293164165 learned_addr learned my addr 172.21.15.42:0/4293164165 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.833+0000 7ff8c4c98700 1 -- 172.21.15.42:0/4293164165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4009550 con 0x7ff8c007ce70 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.833+0000 7ff8c4c98700 1 --2- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c019ccf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009fb0 tx=0x7ff8b4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.833+0000 7ff8bdffb700 1 -- 172.21.15.42:0/4293164165 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4003a40 con 0x7ff8c007ce70 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.833+0000 7ff8bdffb700 1 -- 172.21.15.42:0/4293164165 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff8b4003ba0 con 0x7ff8c007ce70 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.833+0000 7ff8bdffb700 1 -- 172.21.15.42:0/4293164165 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8b401a750 con 0x7ff8c007ce70 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.833+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c019d430 con 0x7ff8c007ce70 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.833+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c019df90 con 0x7ff8c007ce70 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.834+0000 7ff8bdffb700 1 -- 172.21.15.42:0/4293164165 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7ff8b4024b60 con 0x7ff8c007ce70 2023-12-12T09:50:12.039 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.834+0000 7ff8bdffb700 1 --2- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7ff8b0038820 0x7ff8b003ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.834+0000 7ff8bdffb700 1 -- 172.21.15.42:0/4293164165 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff8b404c850 con 0x7ff8c007ce70 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.834+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c0062c00 con 0x7ff8c007ce70 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.835+0000 7ff8bffff700 1 --2- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7ff8b0038820 0x7ff8b003ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.835+0000 7ff8bffff700 1 --2- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7ff8b0038820 0x7ff8b003ace0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac006fd0 tx=0x7ff8ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:05.839+0000 7ff8bdffb700 1 -- 172.21.15.42:0/4293164165 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff8b402d3f0 con 0x7ff8c007ce70 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:06.212+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8c0103510 con 0x7ff8b0038820 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.677+0000 7ff8bdffb700 1 -- 172.21.15.42:0/4293164165 <== mgr.14120 v2:172.21.15.42:6800/1525180467 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7ff8c0103510 con 0x7ff8b0038820 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7ff8b0038820 msgr2=0x7ff8b003ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 --2- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7ff8b0038820 0x7ff8b003ace0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac006fd0 tx=0x7ff8ac006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 msgr2=0x7ff8c019ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 --2- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c019ccf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4009fb0 tx=0x7ff8b4011ad0 comp rx=0 tx=0).stop 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 shutdown_connections 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 --2- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7ff8b0038820 0x7ff8b003ace0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:12.040 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 --2- 172.21.15.42:0/4293164165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8c007ce70 0x7ff8c019ccf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:12.041 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 >> 172.21.15.42:0/4293164165 conn(0x7ff8c01022d0 msgr2=0x7ff8c0102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:12.041 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 shutdown_connections 2023-12-12T09:50:12.041 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:11.680+0000 7ff8c6efc700 1 -- 172.21.15.42:0/4293164165 wait complete. 2023-12-12T09:50:12.041 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mon service... 2023-12-12T09:50:12.959 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:12 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:12.959 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:12 smithi042 ceph-mon[98894]: Added host smithi042 2023-12-12T09:50:12.959 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:12 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:50:13.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-12-12T09:50:13.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.514+0000 7fdd83f5c700 1 Processor -- start 2023-12-12T09:50:13.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.514+0000 7fdd83f5c700 1 -- start start 2023-12-12T09:50:13.481 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.514+0000 7fdd83f5c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c106060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.514+0000 7fdd83f5c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c106630 con 0x7fdd7c105c40 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.514+0000 7fdd81cf8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c106060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.514+0000 7fdd81cf8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c106060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60504/0 (socket says 172.21.15.42:60504) 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.514+0000 7fdd81cf8700 1 -- 172.21.15.42:0/2753747297 learned_addr learned my addr 172.21.15.42:0/2753747297 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.515+0000 7fdd81cf8700 1 -- 172.21.15.42:0/2753747297 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c106e40 con 0x7fdd7c105c40 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.515+0000 7fdd81cf8700 1 --2- 172.21.15.42:0/2753747297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c106060 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd780098a0 tx=0x7fdd78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea91312e1a29d98 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.515+0000 7fdd80cf6700 1 -- 172.21.15.42:0/2753747297 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd78004160 con 0x7fdd7c105c40 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.515+0000 7fdd80cf6700 1 -- 172.21.15.42:0/2753747297 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd780042c0 con 0x7fdd7c105c40 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.515+0000 7fdd80cf6700 1 -- 172.21.15.42:0/2753747297 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd78011a00 con 0x7fdd7c105c40 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.516+0000 7fdd83f5c700 1 -- 172.21.15.42:0/2753747297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 msgr2=0x7fdd7c106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.516+0000 7fdd83f5c700 1 --2- 172.21.15.42:0/2753747297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c106060 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd780098a0 tx=0x7fdd78009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.516+0000 7fdd83f5c700 1 -- 172.21.15.42:0/2753747297 shutdown_connections 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.516+0000 7fdd83f5c700 1 --2- 172.21.15.42:0/2753747297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c106060 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.516+0000 7fdd83f5c700 1 -- 172.21.15.42:0/2753747297 >> 172.21.15.42:0/2753747297 conn(0x7fdd7c1011e0 msgr2=0x7fdd7c103620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:13.482 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.516+0000 7fdd83f5c700 1 -- 172.21.15.42:0/2753747297 shutdown_connections 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.516+0000 7fdd83f5c700 1 -- 172.21.15.42:0/2753747297 wait complete. 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.516+0000 7fdd83f5c700 1 Processor -- start 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.517+0000 7fdd83f5c700 1 -- start start 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.517+0000 7fdd83f5c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.517+0000 7fdd83f5c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c199f50 con 0x7fdd7c105c40 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.517+0000 7fdd81cf8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.517+0000 7fdd81cf8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60506/0 (socket says 172.21.15.42:60506) 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.517+0000 7fdd81cf8700 1 -- 172.21.15.42:0/3325473523 learned_addr learned my addr 172.21.15.42:0/3325473523 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.517+0000 7fdd81cf8700 1 -- 172.21.15.42:0/3325473523 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78009550 con 0x7fdd7c105c40 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.517+0000 7fdd81cf8700 1 --2- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c199a10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009fb0 tx=0x7fdd78011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.517+0000 7fdd72ffd700 1 -- 172.21.15.42:0/3325473523 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd78003a40 con 0x7fdd7c105c40 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.518+0000 7fdd72ffd700 1 -- 172.21.15.42:0/3325473523 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdd78003ba0 con 0x7fdd7c105c40 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.518+0000 7fdd72ffd700 1 -- 172.21.15.42:0/3325473523 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd7801a750 con 0x7fdd7c105c40 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.518+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c19a150 con 0x7fdd7c105c40 2023-12-12T09:50:13.483 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.518+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c19acb0 con 0x7fdd7c105c40 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.518+0000 7fdd72ffd700 1 -- 172.21.15.42:0/3325473523 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fdd7801a980 con 0x7fdd7c105c40 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.519+0000 7fdd72ffd700 1 --2- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd68038820 0x7fdd6803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.519+0000 7fdd72ffd700 1 -- 172.21.15.42:0/3325473523 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdd7804c8c0 con 0x7fdd7c105c40 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.519+0000 7fdd814f7700 1 --2- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd68038820 0x7fdd6803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.519+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7c19a5f0 con 0x7fdd7c105c40 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.519+0000 7fdd814f7700 1 --2- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd68038820 0x7fdd6803ace0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006fd0 tx=0x7fdd6c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.524+0000 7fdd72ffd700 1 -- 172.21.15.42:0/3325473523 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdd7c19a5f0 con 0x7fdd7c105c40 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.928+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fdd7c19a5f0 con 0x7fdd68038820 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.933+0000 7fdd72ffd700 1 -- 172.21.15.42:0/3325473523 <== mgr.14120 v2:172.21.15.42:6800/1525180467 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdd7c19a5f0 con 0x7fdd68038820 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd68038820 msgr2=0x7fdd6803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 --2- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd68038820 0x7fdd6803ace0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006fd0 tx=0x7fdd6c006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 msgr2=0x7fdd7c199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 --2- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c199a10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009fb0 tx=0x7fdd78011ad0 comp rx=0 tx=0).stop 2023-12-12T09:50:13.484 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 shutdown_connections 2023-12-12T09:50:13.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 --2- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd68038820 0x7fdd6803ace0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:13.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 --2- 172.21.15.42:0/3325473523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd7c105c40 0x7fdd7c199a10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:13.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 >> 172.21.15.42:0/3325473523 conn(0x7fdd7c1011e0 msgr2=0x7fdd7c101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:13.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 shutdown_connections 2023-12-12T09:50:13.485 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:12.936+0000 7fdd83f5c700 1 -- 172.21.15.42:0/3325473523 wait complete. 2023-12-12T09:50:13.485 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mgr service... 2023-12-12T09:50:14.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:13 smithi042 ceph-mon[98894]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:14.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:13 smithi042 ceph-mon[98894]: Saving service mon spec with placement count:5 2023-12-12T09:50:14.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:13 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:14.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:13 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-12-12T09:50:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.158+0000 7f84a287a700 1 Processor -- start 2023-12-12T09:50:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.158+0000 7f84a287a700 1 -- start start 2023-12-12T09:50:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.158+0000 7f84a287a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c107110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.158+0000 7f84a287a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c1076e0 con 0x7f849c106cf0 2023-12-12T09:50:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.158+0000 7f849bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c107110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.158+0000 7f849bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c107110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60510/0 (socket says 172.21.15.42:60510) 2023-12-12T09:50:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.158+0000 7f849bfff700 1 -- 172.21.15.42:0/2745272872 learned_addr learned my addr 172.21.15.42:0/2745272872 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:15.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.159+0000 7f849bfff700 1 -- 172.21.15.42:0/2745272872 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c107ef0 con 0x7f849c106cf0 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.159+0000 7f849bfff700 1 --2- 172.21.15.42:0/2745272872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c107110 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f848c0098a0 tx=0x7f848c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46207a90641c6c27 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.159+0000 7f849affd700 1 -- 172.21.15.42:0/2745272872 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f848c004160 con 0x7f849c106cf0 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.159+0000 7f849affd700 1 -- 172.21.15.42:0/2745272872 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f848c0042c0 con 0x7f849c106cf0 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.159+0000 7f849affd700 1 -- 172.21.15.42:0/2745272872 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f848c0119b0 con 0x7f849c106cf0 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.160+0000 7f84a287a700 1 -- 172.21.15.42:0/2745272872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 msgr2=0x7f849c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.160+0000 7f84a287a700 1 --2- 172.21.15.42:0/2745272872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c107110 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f848c0098a0 tx=0x7f848c009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.160+0000 7f84a287a700 1 -- 172.21.15.42:0/2745272872 shutdown_connections 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.160+0000 7f84a287a700 1 --2- 172.21.15.42:0/2745272872 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c107110 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.160+0000 7f84a287a700 1 -- 172.21.15.42:0/2745272872 >> 172.21.15.42:0/2745272872 conn(0x7f849c1022d0 msgr2=0x7f849c1046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.160+0000 7f84a287a700 1 -- 172.21.15.42:0/2745272872 shutdown_connections 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.160+0000 7f84a287a700 1 -- 172.21.15.42:0/2745272872 wait complete. 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.161+0000 7f84a287a700 1 Processor -- start 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.161+0000 7f84a287a700 1 -- start start 2023-12-12T09:50:15.053 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.161+0000 7f84a287a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.161+0000 7f84a287a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c19b020 con 0x7f849c106cf0 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.161+0000 7f849bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.161+0000 7f849bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c19aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60512/0 (socket says 172.21.15.42:60512) 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.161+0000 7f849bfff700 1 -- 172.21.15.42:0/2968616330 learned_addr learned my addr 172.21.15.42:0/2968616330 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.161+0000 7f849bfff700 1 -- 172.21.15.42:0/2968616330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c009550 con 0x7f849c106cf0 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.162+0000 7f849bfff700 1 --2- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c19aae0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f848c006af0 tx=0x7f848c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.162+0000 7f84997fa700 1 -- 172.21.15.42:0/2968616330 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f848c003a40 con 0x7f849c106cf0 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.162+0000 7f84997fa700 1 -- 172.21.15.42:0/2968616330 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f848c003ba0 con 0x7f849c106cf0 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.162+0000 7f84997fa700 1 -- 172.21.15.42:0/2968616330 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f848c019690 con 0x7f849c106cf0 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.162+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849c19b220 con 0x7f849c106cf0 2023-12-12T09:50:15.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.162+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849c19bd80 con 0x7f849c106cf0 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.163+0000 7f84997fa700 1 -- 172.21.15.42:0/2968616330 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f848c01aa60 con 0x7f849c106cf0 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.163+0000 7f84997fa700 1 --2- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f8488038820 0x7f848803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.163+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849c062c00 con 0x7f849c106cf0 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.163+0000 7f84997fa700 1 -- 172.21.15.42:0/2968616330 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f848c04c780 con 0x7f849c106cf0 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.163+0000 7f849b7fe700 1 --2- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f8488038820 0x7f848803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.164+0000 7f849b7fe700 1 --2- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f8488038820 0x7f848803ace0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8484006fd0 tx=0x7f8484006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.168+0000 7f84997fa700 1 -- 172.21.15.42:0/2968616330 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f848c02d430 con 0x7f849c106cf0 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.564+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f849c103530 con 0x7f8488038820 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.569+0000 7f84997fa700 1 -- 172.21.15.42:0/2968616330 <== mgr.14120 v2:172.21.15.42:6800/1525180467 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f849c103530 con 0x7f8488038820 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.571+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f8488038820 msgr2=0x7f848803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.571+0000 7f84a287a700 1 --2- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f8488038820 0x7f848803ace0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8484006fd0 tx=0x7f8484006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:15.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.571+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 msgr2=0x7f849c19aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:15.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.571+0000 7f84a287a700 1 --2- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c19aae0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f848c006af0 tx=0x7f848c011a10 comp rx=0 tx=0).stop 2023-12-12T09:50:15.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.572+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 shutdown_connections 2023-12-12T09:50:15.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.572+0000 7f84a287a700 1 --2- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f8488038820 0x7f848803ace0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:15.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.572+0000 7f84a287a700 1 --2- 172.21.15.42:0/2968616330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f849c106cf0 0x7f849c19aae0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:15.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.572+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 >> 172.21.15.42:0/2968616330 conn(0x7f849c1022d0 msgr2=0x7f849c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:15.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.572+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 shutdown_connections 2023-12-12T09:50:15.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:14.572+0000 7f84a287a700 1 -- 172.21.15.42:0/2968616330 wait complete. 2023-12-12T09:50:15.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:15 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:15.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:15 smithi042 ceph-mon[98894]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:15.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:15 smithi042 ceph-mon[98894]: Saving service mgr spec with placement count:2 2023-12-12T09:50:15.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:15 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:16.601 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.536+0000 7fd8f0734700 1 Processor -- start 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.536+0000 7fd8f0734700 1 -- start start 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.536+0000 7fd8f0734700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e8109b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.536+0000 7fd8f0734700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e807ae60 con 0x7fd8e81077a0 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.537+0000 7fd8ee4d0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e8109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.537+0000 7fd8ee4d0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e8109b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60516/0 (socket says 172.21.15.42:60516) 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.537+0000 7fd8ee4d0700 1 -- 172.21.15.42:0/344960982 learned_addr learned my addr 172.21.15.42:0/344960982 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.537+0000 7fd8ee4d0700 1 -- 172.21.15.42:0/344960982 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e810a0d0 con 0x7fd8e81077a0 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.537+0000 7fd8ee4d0700 1 --2- 172.21.15.42:0/344960982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e8109b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40098a0 tx=0x7fd8e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf60d5237f122bf8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.537+0000 7fd8ed4ce700 1 -- 172.21.15.42:0/344960982 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e4004160 con 0x7fd8e81077a0 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.537+0000 7fd8ed4ce700 1 -- 172.21.15.42:0/344960982 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e40042c0 con 0x7fd8e81077a0 2023-12-12T09:50:16.602 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.537+0000 7fd8ed4ce700 1 -- 172.21.15.42:0/344960982 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e40119b0 con 0x7fd8e81077a0 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.538+0000 7fd8f0734700 1 -- 172.21.15.42:0/344960982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 msgr2=0x7fd8e8109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.538+0000 7fd8f0734700 1 --2- 172.21.15.42:0/344960982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e8109b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40098a0 tx=0x7fd8e4009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.538+0000 7fd8f0734700 1 -- 172.21.15.42:0/344960982 shutdown_connections 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.538+0000 7fd8f0734700 1 --2- 172.21.15.42:0/344960982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e8109b90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.538+0000 7fd8f0734700 1 -- 172.21.15.42:0/344960982 >> 172.21.15.42:0/344960982 conn(0x7fd8e81011d0 msgr2=0x7fd8e8103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.538+0000 7fd8f0734700 1 -- 172.21.15.42:0/344960982 shutdown_connections 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.539+0000 7fd8f0734700 1 -- 172.21.15.42:0/344960982 wait complete. 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.539+0000 7fd8f0734700 1 Processor -- start 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.539+0000 7fd8f0734700 1 -- start start 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.539+0000 7fd8f0734700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8f0734700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e819af20 con 0x7fd8e81077a0 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8ee4d0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8ee4d0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60518/0 (socket says 172.21.15.42:60518) 2023-12-12T09:50:16.603 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8ee4d0700 1 -- 172.21.15.42:0/3039104098 learned_addr learned my addr 172.21.15.42:0/3039104098 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8ee4d0700 1 -- 172.21.15.42:0/3039104098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4009550 con 0x7fd8e81077a0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8ee4d0700 1 --2- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e819a9e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4011670 tx=0x7fd8e4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8df7fe700 1 -- 172.21.15.42:0/3039104098 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e4003b20 con 0x7fd8e81077a0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e819b120 con 0x7fd8e81077a0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8df7fe700 1 -- 172.21.15.42:0/3039104098 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8e4003c80 con 0x7fd8e81077a0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8df7fe700 1 -- 172.21.15.42:0/3039104098 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8e4019770 con 0x7fd8e81077a0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.540+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e819b5c0 con 0x7fd8e81077a0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.542+0000 7fd8df7fe700 1 -- 172.21.15.42:0/3039104098 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7fd8e4019a30 con 0x7fd8e81077a0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.542+0000 7fd8df7fe700 1 --2- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fd8d4038820 0x7fd8d403ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.542+0000 7fd8df7fe700 1 -- 172.21.15.42:0/3039104098 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd8e404cad0 con 0x7fd8e81077a0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.542+0000 7fd8edccf700 1 --2- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fd8d4038820 0x7fd8d403ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.542+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e8068600 con 0x7fd8e81077a0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.542+0000 7fd8edccf700 1 --2- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fd8d4038820 0x7fd8d403ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8006fd0 tx=0x7fd8d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:16.604 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.547+0000 7fd8df7fe700 1 -- 172.21.15.42:0/3039104098 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd8e4018b80 con 0x7fd8e81077a0 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.920+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fd8e819dfe0 con 0x7fd8e81077a0 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.922+0000 7fd8df7fe700 1 -- 172.21.15.42:0/3039104098 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fd8e4023e60 con 0x7fd8e81077a0 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.924+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fd8d4038820 msgr2=0x7fd8d403ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.924+0000 7fd8f0734700 1 --2- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fd8d4038820 0x7fd8d403ace0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd8d8006fd0 tx=0x7fd8d8006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.924+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 msgr2=0x7fd8e819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.924+0000 7fd8f0734700 1 --2- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e819a9e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4011670 tx=0x7fd8e4011a30 comp rx=0 tx=0).stop 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.924+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 shutdown_connections 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.924+0000 7fd8f0734700 1 --2- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fd8d4038820 0x7fd8d403ace0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.925+0000 7fd8f0734700 1 --2- 172.21.15.42:0/3039104098 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8e81077a0 0x7fd8e819a9e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.925+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 >> 172.21.15.42:0/3039104098 conn(0x7fd8e81011d0 msgr2=0x7fd8e8101cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.925+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 shutdown_connections 2023-12-12T09:50:16.605 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:15.925+0000 7fd8f0734700 1 -- 172.21.15.42:0/3039104098 wait complete. 2023-12-12T09:50:17.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:16 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3039104098' entity='client.admin' 2023-12-12T09:50:17.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:16 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:18.051 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.142+0000 7f0890465700 1 Processor -- start 2023-12-12T09:50:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.142+0000 7f0890465700 1 -- start start 2023-12-12T09:50:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.143+0000 7f0890465700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f0888106030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.143+0000 7f0890465700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0888106600 con 0x7f0888105c10 2023-12-12T09:50:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.143+0000 7f088e201700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f0888106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.143+0000 7f088e201700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f0888106030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60522/0 (socket says 172.21.15.42:60522) 2023-12-12T09:50:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.143+0000 7f088e201700 1 -- 172.21.15.42:0/454450373 learned_addr learned my addr 172.21.15.42:0/454450373 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.144+0000 7f088e201700 1 -- 172.21.15.42:0/454450373 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0888106e10 con 0x7f0888105c10 2023-12-12T09:50:18.052 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.144+0000 7f088e201700 1 --2- 172.21.15.42:0/454450373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f0888106030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08840098a0 tx=0x7f0884009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee0b73ccabb8cbc server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:18.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.144+0000 7f088d1ff700 1 -- 172.21.15.42:0/454450373 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0884004160 con 0x7f0888105c10 2023-12-12T09:50:18.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.144+0000 7f088d1ff700 1 -- 172.21.15.42:0/454450373 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08840042c0 con 0x7f0888105c10 2023-12-12T09:50:18.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.144+0000 7f088d1ff700 1 -- 172.21.15.42:0/454450373 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08840119b0 con 0x7f0888105c10 2023-12-12T09:50:18.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.145+0000 7f0890465700 1 -- 172.21.15.42:0/454450373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 msgr2=0x7f0888106030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:18.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.145+0000 7f0890465700 1 --2- 172.21.15.42:0/454450373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f0888106030 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f08840098a0 tx=0x7f0884009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:18.054 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.145+0000 7f0890465700 1 -- 172.21.15.42:0/454450373 shutdown_connections 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.145+0000 7f0890465700 1 --2- 172.21.15.42:0/454450373 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f0888106030 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.145+0000 7f0890465700 1 -- 172.21.15.42:0/454450373 >> 172.21.15.42:0/454450373 conn(0x7f08881011d0 msgr2=0x7f08881035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.145+0000 7f0890465700 1 -- 172.21.15.42:0/454450373 shutdown_connections 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.145+0000 7f0890465700 1 -- 172.21.15.42:0/454450373 wait complete. 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.146+0000 7f0890465700 1 Processor -- start 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.146+0000 7f0890465700 1 -- start start 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.146+0000 7f0890465700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f088819b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.146+0000 7f0890465700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088807abb0 con 0x7f0888105c10 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f088e201700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f088819b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f088e201700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f088819b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60524/0 (socket says 172.21.15.42:60524) 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f088e201700 1 -- 172.21.15.42:0/4152897346 learned_addr learned my addr 172.21.15.42:0/4152897346 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f088e201700 1 -- 172.21.15.42:0/4152897346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0884009550 con 0x7f0888105c10 2023-12-12T09:50:18.055 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f088e201700 1 --2- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f088819b540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0884003e60 tx=0x7f0884011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f087f7fe700 1 -- 172.21.15.42:0/4152897346 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0884003a60 con 0x7f0888105c10 2023-12-12T09:50:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f087f7fe700 1 -- 172.21.15.42:0/4152897346 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0884003bc0 con 0x7f0888105c10 2023-12-12T09:50:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088807adb0 con 0x7f0888105c10 2023-12-12T09:50:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f087f7fe700 1 -- 172.21.15.42:0/4152897346 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08840196c0 con 0x7f0888105c10 2023-12-12T09:50:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.147+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088819bcd0 con 0x7f0888105c10 2023-12-12T09:50:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.148+0000 7f087f7fe700 1 -- 172.21.15.42:0/4152897346 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f08840199c0 con 0x7f0888105c10 2023-12-12T09:50:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.149+0000 7f087f7fe700 1 --2- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f0874038800 0x7f087403acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.149+0000 7f087f7fe700 1 -- 172.21.15.42:0/4152897346 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f088404da40 con 0x7f0888105c10 2023-12-12T09:50:18.056 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.149+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0888062c00 con 0x7f0888105c10 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.149+0000 7f088da00700 1 --2- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f0874038800 0x7f087403acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.149+0000 7f088da00700 1 --2- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f0874038800 0x7f087403acc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0878006fd0 tx=0x7f0878006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.154+0000 7f087f7fe700 1 -- 172.21.15.42:0/4152897346 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0884019cc0 con 0x7f0888105c10 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.522+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f0888067940 con 0x7f0888105c10 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.525+0000 7f087f7fe700 1 -- 172.21.15.42:0/4152897346 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f0884018350 con 0x7f0888105c10 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f0874038800 msgr2=0x7f087403acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 --2- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f0874038800 0x7f087403acc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0878006fd0 tx=0x7f0878006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 msgr2=0x7f088819b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 --2- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f088819b540 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0884003e60 tx=0x7f0884011a30 comp rx=0 tx=0).stop 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 shutdown_connections 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 --2- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f0874038800 0x7f087403acc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 --2- 172.21.15.42:0/4152897346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0888105c10 0x7f088819b540 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:18.057 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 >> 172.21.15.42:0/4152897346 conn(0x7f08881011d0 msgr2=0x7f0888101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:18.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 shutdown_connections 2023-12-12T09:50:18.058 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:17.528+0000 7f0890465700 1 -- 172.21.15.42:0/4152897346 wait complete. 2023-12-12T09:50:18.058 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling the dashboard module... 2023-12-12T09:50:18.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:18 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/4152897346' entity='client.admin' 2023-12-12T09:50:18.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:18 smithi042 ceph-mon[98894]: from='mgr.14120 172.21.15.42:0/1997637478' entity='mgr.x' 2023-12-12T09:50:19.741 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:19 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: ignoring --setuser ceph since I am not root 2023-12-12T09:50:19.741 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:19 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: ignoring --setgroup ceph since I am not root 2023-12-12T09:50:19.741 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:19 smithi042 ceph-mon[98894]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:50:19.741 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:19 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1792197001' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.552+0000 7f257fbfd700 1 Processor -- start 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.552+0000 7f257fbfd700 1 -- start start 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.552+0000 7f257fbfd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f2578106020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.552+0000 7f257fbfd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25781065f0 con 0x7f2578105c00 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.552+0000 7f257d999700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f2578106020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.552+0000 7f257d999700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f2578106020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60528/0 (socket says 172.21.15.42:60528) 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.552+0000 7f257d999700 1 -- 172.21.15.42:0/1526548975 learned_addr learned my addr 172.21.15.42:0/1526548975 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.553+0000 7f257d999700 1 -- 172.21.15.42:0/1526548975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578106e00 con 0x7f2578105c00 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.553+0000 7f257d999700 1 --2- 172.21.15.42:0/1526548975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f2578106020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f257400bb40 tx=0x7f257400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874ad0a66467d9f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:19.995 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.553+0000 7f257c997700 1 -- 172.21.15.42:0/1526548975 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f257400dbe0 con 0x7f2578105c00 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.553+0000 7f257c997700 1 -- 172.21.15.42:0/1526548975 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25740044f0 con 0x7f2578105c00 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.553+0000 7f257c997700 1 -- 172.21.15.42:0/1526548975 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2574003e60 con 0x7f2578105c00 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.554+0000 7f257fbfd700 1 -- 172.21.15.42:0/1526548975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 msgr2=0x7f2578106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.554+0000 7f257fbfd700 1 --2- 172.21.15.42:0/1526548975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f2578106020 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f257400bb40 tx=0x7f257400be50 comp rx=0 tx=0).stop 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.554+0000 7f257fbfd700 1 -- 172.21.15.42:0/1526548975 shutdown_connections 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.554+0000 7f257fbfd700 1 --2- 172.21.15.42:0/1526548975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f2578106020 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.554+0000 7f257fbfd700 1 -- 172.21.15.42:0/1526548975 >> 172.21.15.42:0/1526548975 conn(0x7f2578101200 msgr2=0x7f2578103620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.554+0000 7f257fbfd700 1 -- 172.21.15.42:0/1526548975 shutdown_connections 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.554+0000 7f257fbfd700 1 -- 172.21.15.42:0/1526548975 wait complete. 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.555+0000 7f257fbfd700 1 Processor -- start 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.555+0000 7f257fbfd700 1 -- start start 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.555+0000 7f257fbfd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f257819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.555+0000 7f257fbfd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257819d250 con 0x7f2578105c00 2023-12-12T09:50:19.996 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.556+0000 7f257d999700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f257819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.556+0000 7f257d999700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f257819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60530/0 (socket says 172.21.15.42:60530) 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.556+0000 7f257d999700 1 -- 172.21.15.42:0/1792197001 learned_addr learned my addr 172.21.15.42:0/1792197001 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.556+0000 7f257d999700 1 -- 172.21.15.42:0/1792197001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257400b7f0 con 0x7f2578105c00 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.556+0000 7f257d999700 1 --2- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f257819cd10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25740104d0 tx=0x7f2574004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.557+0000 7f256effd700 1 -- 172.21.15.42:0/1792197001 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2574010760 con 0x7f2578105c00 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.557+0000 7f256effd700 1 -- 172.21.15.42:0/1792197001 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2574010d80 con 0x7f2578105c00 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.557+0000 7f256effd700 1 -- 172.21.15.42:0/1792197001 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2574019c90 con 0x7f2578105c00 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.557+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257819d450 con 0x7f2578105c00 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.557+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257819d890 con 0x7f2578105c00 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.558+0000 7f256effd700 1 -- 172.21.15.42:0/1792197001 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) v1 ==== 45580+0+0 (secure 0 0 0) 0x7f25740194f0 con 0x7f2578105c00 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.558+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2578067940 con 0x7f2578105c00 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.558+0000 7f256effd700 1 --2- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f2564038870 0x7f256403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.559+0000 7f256effd700 1 -- 172.21.15.42:0/1792197001 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f257404c6d0 con 0x7f2578105c00 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.559+0000 7f257d198700 1 --2- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f2564038870 0x7f256403ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:19.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.559+0000 7f257d198700 1 --2- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f2564038870 0x7f256403ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2568006fd0 tx=0x7f2568006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:18.564+0000 7f256effd700 1 -- 172.21.15.42:0/1792197001 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2574028530 con 0x7f2578105c00 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.051+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f2578079f80 con 0x7f2578105c00 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.527+0000 7f256effd700 1 -- 172.21.15.42:0/1792197001 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f2574028350 con 0x7f2578105c00 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.529+0000 7f256effd700 1 -- 172.21.15.42:0/1792197001 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f25740293f0 con 0x7f2578105c00 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f2564038870 msgr2=0x7f256403ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 --2- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f2564038870 0x7f256403ad30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2568006fd0 tx=0x7f2568006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 msgr2=0x7f257819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 --2- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f257819cd10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25740104d0 tx=0x7f2574004020 comp rx=0 tx=0).stop 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 shutdown_connections 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 --2- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7f2564038870 0x7f256403ad30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 --2- 172.21.15.42:0/1792197001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2578105c00 0x7f257819cd10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 >> 172.21.15.42:0/1792197001 conn(0x7f2578101200 msgr2=0x7f2578193450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 shutdown_connections 2023-12-12T09:50:19.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:19.530+0000 7f257fbfd700 1 -- 172.21.15.42:0/1792197001 wait complete. 2023-12-12T09:50:20.074 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:19 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:19.739+0000 7f8e8eff2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:50:20.074 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:19 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:19.836+0000 7f8e8eff2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:50:20.074 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:19 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:19.945+0000 7f8e8eff2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:50:20.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:20 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:20.072+0000 7f8e8eff2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:50:20.344 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:20 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:20.281+0000 7f8e8eff2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:50:20.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:20 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1792197001' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-12T09:50:20.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:20 smithi042 ceph-mon[98894]: mgrmap e9: x(active, since 21s) 2023-12-12T09:50:21.030 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:20 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:20.768+0000 7f8e8eff2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:50:21.030 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:20 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:20.920+0000 7f8e8eff2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:50:21.402 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:21 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:21.028+0000 7f8e8eff2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:50:21.403 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:21 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:21.145+0000 7f8e8eff2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:50:21.457 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2023-12-12T09:50:21.457 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-12-12T09:50:21.457 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-12T09:50:21.457 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-12T09:50:21.457 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-12T09:50:21.457 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2023-12-12T09:50:21.457 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.590+0000 7fc094107700 1 Processor -- start 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.590+0000 7fc094107700 1 -- start start 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.591+0000 7fc094107700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c1059b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.591+0000 7fc094107700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c07ae60 con 0x7fc08c105590 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.591+0000 7fc091ea3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c1059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.591+0000 7fc091ea3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c1059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60544/0 (socket says 172.21.15.42:60544) 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.591+0000 7fc091ea3700 1 -- 172.21.15.42:0/1466457907 learned_addr learned my addr 172.21.15.42:0/1466457907 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.591+0000 7fc091ea3700 1 -- 172.21.15.42:0/1466457907 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c105ef0 con 0x7fc08c105590 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.591+0000 7fc091ea3700 1 --2- 172.21.15.42:0/1466457907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c1059b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0098a0 tx=0x7fc07c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fec847ae22af79b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.591+0000 7fc090ea1700 1 -- 172.21.15.42:0/1466457907 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc07c004160 con 0x7fc08c105590 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.592+0000 7fc090ea1700 1 -- 172.21.15.42:0/1466457907 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c0042c0 con 0x7fc08c105590 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.592+0000 7fc090ea1700 1 -- 172.21.15.42:0/1466457907 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc07c0119b0 con 0x7fc08c105590 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.592+0000 7fc094107700 1 -- 172.21.15.42:0/1466457907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 msgr2=0x7fc08c1059b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.592+0000 7fc094107700 1 --2- 172.21.15.42:0/1466457907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c1059b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0098a0 tx=0x7fc07c009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:21.458 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.593+0000 7fc094107700 1 -- 172.21.15.42:0/1466457907 shutdown_connections 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.593+0000 7fc094107700 1 --2- 172.21.15.42:0/1466457907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c1059b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.593+0000 7fc094107700 1 -- 172.21.15.42:0/1466457907 >> 172.21.15.42:0/1466457907 conn(0x7fc08c1011d0 msgr2=0x7fc08c103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.593+0000 7fc094107700 1 -- 172.21.15.42:0/1466457907 shutdown_connections 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.593+0000 7fc094107700 1 -- 172.21.15.42:0/1466457907 wait complete. 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.593+0000 7fc094107700 1 Processor -- start 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.593+0000 7fc094107700 1 -- start start 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc094107700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc094107700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c19d140 con 0x7fc08c105590 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc091ea3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc091ea3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60546/0 (socket says 172.21.15.42:60546) 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc091ea3700 1 -- 172.21.15.42:0/1654758861 learned_addr learned my addr 172.21.15.42:0/1654758861 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc091ea3700 1 -- 172.21.15.42:0/1654758861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c009550 con 0x7fc08c105590 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc091ea3700 1 --2- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c19cc00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006af0 tx=0x7fc07c011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc082ffd700 1 -- 172.21.15.42:0/1654758861 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc07c003a40 con 0x7fc08c105590 2023-12-12T09:50:21.459 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc082ffd700 1 -- 172.21.15.42:0/1654758861 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc07c003ba0 con 0x7fc08c105590 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc082ffd700 1 -- 172.21.15.42:0/1654758861 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc07c0196c0 con 0x7fc08c105590 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.594+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c19d340 con 0x7fc08c105590 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.595+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c19d7e0 con 0x7fc08c105590 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.595+0000 7fc082ffd700 1 -- 172.21.15.42:0/1654758861 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7fc07c019930 con 0x7fc08c105590 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.596+0000 7fc082ffd700 1 --2- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fc078038870 0x7fc07803ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.596+0000 7fc082ffd700 1 -- 172.21.15.42:0/1654758861 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc07c04d9f0 con 0x7fc08c105590 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.596+0000 7fc0916a2700 1 -- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fc078038870 msgr2=0x7fc07803ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1525180467 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.596+0000 7fc0916a2700 1 --2- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fc078038870 0x7fc07803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.596+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c196900 con 0x7fc08c105590 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.601+0000 7fc082ffd700 1 -- 172.21.15.42:0/1654758861 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc07c019e60 con 0x7fc08c105590 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.797+0000 7fc0916a2700 1 -- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fc078038870 msgr2=0x7fc07803ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1525180467 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:20.797+0000 7fc0916a2700 1 --2- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fc078038870 0x7fc07803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.079+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fc08c106da0 con 0x7fc08c105590 2023-12-12T09:50:21.460 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.080+0000 7fc082ffd700 1 -- 172.21.15.42:0/1654758861 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fc07c0172b0 con 0x7fc08c105590 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fc078038870 msgr2=0x7fc07803ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 --2- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fc078038870 0x7fc07803ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 msgr2=0x7fc08c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 --2- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c19cc00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc07c006af0 tx=0x7fc07c011a10 comp rx=0 tx=0).stop 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 shutdown_connections 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 --2- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fc078038870 0x7fc07803ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 --2- 172.21.15.42:0/1654758861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc08c105590 0x7fc08c19cc00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 >> 172.21.15.42:0/1654758861 conn(0x7fc08c1011d0 msgr2=0x7fc08c193a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 shutdown_connections 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:21.082+0000 7fc094107700 1 -- 172.21.15.42:0/1654758861 wait complete. 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2023-12-12T09:50:21.461 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 9... 2023-12-12T09:50:21.744 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:21 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1654758861' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-12T09:50:21.745 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:21 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:21.672+0000 7f8e8eff2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:50:22.902 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:22 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:22.465+0000 7f8e8eff2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:50:22.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:22 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:22.563+0000 7f8e8eff2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:50:23.882 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:23 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:23.544+0000 7f8e8eff2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:50:23.882 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:23 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:23.655+0000 7f8e8eff2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:50:24.152 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:23 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:23.880+0000 7f8e8eff2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:50:24.652 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:24 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:24.378+0000 7f8e8eff2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:50:24.652 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:24 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:24.483+0000 7f8e8eff2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:50:24.999 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:24 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:24.666+0000 7f8e8eff2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:50:25.402 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:24 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:24.998+0000 7f8e8eff2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:50:26.152 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:25 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:25.820+0000 7f8e8eff2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:50:26.153 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:25 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:25.978+0000 7f8e8eff2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:50:26.644 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:26 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:26.228+0000 7f8e8eff2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:50:26.644 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:26 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:26.364+0000 7f8e8eff2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:50:26.902 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:26 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:26.641+0000 7f8e8eff2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:50:28.153 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:50:27.713+0000 7f8e8eff2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:50:28.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: Active manager daemon x restarted 2023-12-12T09:50:28.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: Activating manager daemon x 2023-12-12T09:50:28.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: osdmap e3: 0 total, 0 up, 0 in 2023-12-12T09:50:28.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: mgrmap e10: x(active, starting, since 0.00330959s) 2023-12-12T09:50:28.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:50:28.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-12T09:50:28.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-12T09:50:28.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-12T09:50:28.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-12T09:50:28.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: Manager daemon x is now available 2023-12-12T09:50:28.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:50:28.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:50:29.035 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-12T09:50:29.035 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-12T09:50:29.035 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:28 smithi042 ceph-mon[98894]: [12/Dec/2023:09:50:28] ENGINE Bus STARTING 2023-12-12T09:50:29.035 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:28 smithi042 ceph-mon[98894]: [12/Dec/2023:09:50:28] ENGINE Serving on http://172.21.15.42:8765 2023-12-12T09:50:29.035 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:28 smithi042 ceph-mon[98894]: mgrmap e11: x(active, since 1.00664s) 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.001+0000 7fdd990ba700 1 Processor -- start 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.001+0000 7fdd990ba700 1 -- start start 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.001+0000 7fdd990ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9407b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.001+0000 7fdd990ba700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9407b860 con 0x7fdd9407ce30 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.001+0000 7fdd92d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9407b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.001+0000 7fdd92d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9407b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60554/0 (socket says 172.21.15.42:60554) 2023-12-12T09:50:29.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.001+0000 7fdd92d9d700 1 -- 172.21.15.42:0/1300455683 learned_addr learned my addr 172.21.15.42:0/1300455683 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.002+0000 7fdd92d9d700 1 -- 172.21.15.42:0/1300455683 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9407b9a0 con 0x7fdd9407ce30 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.002+0000 7fdd92d9d700 1 --2- 172.21.15.42:0/1300455683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9407b290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd840098a0 tx=0x7fdd84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8808b3b5c67c2a0b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.002+0000 7fdd91d9b700 1 -- 172.21.15.42:0/1300455683 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd84004160 con 0x7fdd9407ce30 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.002+0000 7fdd91d9b700 1 -- 172.21.15.42:0/1300455683 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd840042c0 con 0x7fdd9407ce30 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.002+0000 7fdd91d9b700 1 -- 172.21.15.42:0/1300455683 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd840119b0 con 0x7fdd9407ce30 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.003+0000 7fdd990ba700 1 -- 172.21.15.42:0/1300455683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 msgr2=0x7fdd9407b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.003+0000 7fdd990ba700 1 --2- 172.21.15.42:0/1300455683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9407b290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd840098a0 tx=0x7fdd84009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.003+0000 7fdd990ba700 1 -- 172.21.15.42:0/1300455683 shutdown_connections 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.003+0000 7fdd990ba700 1 --2- 172.21.15.42:0/1300455683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9407b290 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.003+0000 7fdd990ba700 1 -- 172.21.15.42:0/1300455683 >> 172.21.15.42:0/1300455683 conn(0x7fdd941022e0 msgr2=0x7fdd94104720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.003+0000 7fdd990ba700 1 -- 172.21.15.42:0/1300455683 shutdown_connections 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.003+0000 7fdd990ba700 1 -- 172.21.15.42:0/1300455683 wait complete. 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.004+0000 7fdd990ba700 1 Processor -- start 2023-12-12T09:50:29.121 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.004+0000 7fdd990ba700 1 -- start start 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.004+0000 7fdd990ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.004+0000 7fdd990ba700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9419d140 con 0x7fdd9407ce30 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.004+0000 7fdd92d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.004+0000 7fdd92d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60556/0 (socket says 172.21.15.42:60556) 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.004+0000 7fdd92d9d700 1 -- 172.21.15.42:0/3860152919 learned_addr learned my addr 172.21.15.42:0/3860152919 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.004+0000 7fdd92d9d700 1 -- 172.21.15.42:0/3860152919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84009550 con 0x7fdd9407ce30 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.005+0000 7fdd92d9d700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9419cc00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd84011670 tx=0x7fdd84011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.005+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd84003b20 con 0x7fdd9407ce30 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.005+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd84003c80 con 0x7fdd9407ce30 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.005+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9419d340 con 0x7fdd9407ce30 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.005+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd84019770 con 0x7fdd9407ce30 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.005+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9419d7e0 con 0x7fdd9407ce30 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.006+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) v1 ==== 45593+0+0 (secure 0 0 0) 0x7fdd84019a30 con 0x7fdd9407ce30 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.006+0000 7fdd7bfff700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 0x7fdd7c03ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:29.122 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.006+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 --> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd9410d180 con 0x7fdd7c038870 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.006+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdd8404daa0 con 0x7fdd9407ce30 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.006+0000 7fdd9259c700 1 -- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 msgr2=0x7fdd7c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1525180467 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.006+0000 7fdd9259c700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 0x7fdd7c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.207+0000 7fdd9259c700 1 -- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 msgr2=0x7fdd7c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1525180467 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.207+0000 7fdd9259c700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 0x7fdd7c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.608+0000 7fdd9259c700 1 -- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 msgr2=0x7fdd7c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1525180467 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:22.608+0000 7fdd9259c700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 0x7fdd7c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:23.409+0000 7fdd9259c700 1 -- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 msgr2=0x7fdd7c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1525180467 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:23.409+0000 7fdd9259c700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 0x7fdd7c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:25.010+0000 7fdd9259c700 1 -- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 msgr2=0x7fdd7c03ad30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/1525180467 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:25.010+0000 7fdd9259c700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 0x7fdd7c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:27.728+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 10) v1 ==== 45360+0+0 (secure 0 0 0) 0x7fdd8400b9a0 con 0x7fdd9407ce30 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:27.728+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 msgr2=0x7fdd7c03ad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:50:29.123 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:27.728+0000 7fdd7bfff700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 0x7fdd7c03ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.730+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 11) v1 ==== 45487+0+0 (secure 0 0 0) 0x7fdd8404e770 con 0x7fdd9407ce30 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.730+0000 7fdd7bfff700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdd7c03bfd0 0x7fdd7c03e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.730+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd9410d180 con 0x7fdd7c03bfd0 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.731+0000 7fdd9259c700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdd7c03bfd0 0x7fdd7c03e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.731+0000 7fdd9259c700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdd7c03bfd0 0x7fdd7c03e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdd88003a10 tx=0x7fdd880092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.732+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fdd9410d180 con 0x7fdd7c03bfd0 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.744+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fdd9410d2f0 con 0x7fdd7c03bfd0 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.744+0000 7fdd7bfff700 1 -- 172.21.15.42:0/3860152919 <== mgr.14152 v2:172.21.15.42:6800/2637927159 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fdd9410d2f0 con 0x7fdd7c03bfd0 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdd7c03bfd0 msgr2=0x7fdd7c03e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdd7c03bfd0 0x7fdd7c03e3c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdd88003a10 tx=0x7fdd880092b0 comp rx=0 tx=0).stop 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 msgr2=0x7fdd9419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9419cc00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdd84011670 tx=0x7fdd84011a30 comp rx=0 tx=0).stop 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 shutdown_connections 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdd9407ce30 0x7fdd9419cc00 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:29.124 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdd7c03bfd0 0x7fdd7c03e3c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:29.125 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 --2- 172.21.15.42:0/3860152919 >> [v2:172.21.15.42:6800/1525180467,v1:172.21.15.42:6801/1525180467] conn(0x7fdd7c038870 0x7fdd7c03ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:29.125 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 >> 172.21.15.42:0/3860152919 conn(0x7fdd941022e0 msgr2=0x7fdd94102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:29.125 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 shutdown_connections 2023-12-12T09:50:29.125 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:28.745+0000 7fdd990ba700 1 -- 172.21.15.42:0/3860152919 wait complete. 2023-12-12T09:50:29.125 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 9 is available 2023-12-12T09:50:29.125 INFO:teuthology.orchestra.run.smithi042.stdout:Generating a dashboard self-signed certificate... 2023-12-12T09:50:30.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:29 smithi042 ceph-mon[98894]: [12/Dec/2023:09:50:28] ENGINE Serving on https://172.21.15.42:7150 2023-12-12T09:50:30.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:29 smithi042 ceph-mon[98894]: [12/Dec/2023:09:50:28] ENGINE Bus STARTED 2023-12-12T09:50:30.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:29 smithi042 ceph-mon[98894]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-12T09:50:30.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:29 smithi042 ceph-mon[98894]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-12T09:50:30.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:30.765 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-12-12T09:50:30.765 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.805+0000 7f8501f9a700 1 Processor -- start 2023-12-12T09:50:30.765 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.805+0000 7f8501f9a700 1 -- start start 2023-12-12T09:50:30.765 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.805+0000 7f8501f9a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc1059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:30.765 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.805+0000 7f8501f9a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc07ae20 con 0x7f84fc1055d0 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.805+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc1059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.805+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc1059f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60610/0 (socket says 172.21.15.42:60610) 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.805+0000 7f84fb7fe700 1 -- 172.21.15.42:0/4082074936 learned_addr learned my addr 172.21.15.42:0/4082074936 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.806+0000 7f84fb7fe700 1 -- 172.21.15.42:0/4082074936 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc105f30 con 0x7f84fc1055d0 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.806+0000 7f84fb7fe700 1 --2- 172.21.15.42:0/4082074936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc1059f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0098a0 tx=0x7f84ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8813b401ead165b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.806+0000 7f84fa7fc700 1 -- 172.21.15.42:0/4082074936 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84ec004160 con 0x7f84fc1055d0 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.806+0000 7f84fa7fc700 1 -- 172.21.15.42:0/4082074936 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ec0042c0 con 0x7f84fc1055d0 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.806+0000 7f84fa7fc700 1 -- 172.21.15.42:0/4082074936 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84ec0119b0 con 0x7f84fc1055d0 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.807+0000 7f8501f9a700 1 -- 172.21.15.42:0/4082074936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 msgr2=0x7f84fc1059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.807+0000 7f8501f9a700 1 --2- 172.21.15.42:0/4082074936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc1059f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0098a0 tx=0x7f84ec009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.807+0000 7f8501f9a700 1 -- 172.21.15.42:0/4082074936 shutdown_connections 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.807+0000 7f8501f9a700 1 --2- 172.21.15.42:0/4082074936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc1059f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:30.766 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.807+0000 7f8501f9a700 1 -- 172.21.15.42:0/4082074936 >> 172.21.15.42:0/4082074936 conn(0x7f84fc1011d0 msgr2=0x7f84fc103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.807+0000 7f8501f9a700 1 -- 172.21.15.42:0/4082074936 shutdown_connections 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.807+0000 7f8501f9a700 1 -- 172.21.15.42:0/4082074936 wait complete. 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.808+0000 7f8501f9a700 1 Processor -- start 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.808+0000 7f8501f9a700 1 -- start start 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.808+0000 7f8501f9a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc19cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.808+0000 7f8501f9a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc19d160 con 0x7f84fc1055d0 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.808+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc19cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.809+0000 7f84fb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc19cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60612/0 (socket says 172.21.15.42:60612) 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.809+0000 7f84fb7fe700 1 -- 172.21.15.42:0/2510345322 learned_addr learned my addr 172.21.15.42:0/2510345322 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.809+0000 7f84fb7fe700 1 -- 172.21.15.42:0/2510345322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec009550 con 0x7f84fc1055d0 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.809+0000 7f84fb7fe700 1 --2- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc19cc20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84ec011670 tx=0x7f84ec011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.809+0000 7f84f8ff9700 1 -- 172.21.15.42:0/2510345322 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84ec003b20 con 0x7f84fc1055d0 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.809+0000 7f84f8ff9700 1 -- 172.21.15.42:0/2510345322 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84ec003c80 con 0x7f84fc1055d0 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.809+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc19d360 con 0x7f84fc1055d0 2023-12-12T09:50:30.767 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.809+0000 7f84f8ff9700 1 -- 172.21.15.42:0/2510345322 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84ec019770 con 0x7f84fc1055d0 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.809+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc19d800 con 0x7f84fc1055d0 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.810+0000 7f84f8ff9700 1 -- 172.21.15.42:0/2510345322 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f84ec019a50 con 0x7f84fc1055d0 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.811+0000 7f84f8ff9700 1 --2- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f84e4038870 0x7f84e403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.811+0000 7f84f8ff9700 1 -- 172.21.15.42:0/2510345322 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f84ec04cbd0 con 0x7f84fc1055d0 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.811+0000 7f84faffd700 1 --2- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f84e4038870 0x7f84e403ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.811+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc196950 con 0x7f84fc1055d0 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.811+0000 7f84faffd700 1 --2- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f84e4038870 0x7f84e403ad30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f84f0006fd0 tx=0x7f84f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:29.816+0000 7f84f8ff9700 1 -- 172.21.15.42:0/2510345322 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f84ec018800 con 0x7f84fc1055d0 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.196+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f84fc061a10 con 0x7f84e4038870 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.274+0000 7f84f8ff9700 1 -- 172.21.15.42:0/2510345322 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f84fc061a10 con 0x7f84e4038870 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f84e4038870 msgr2=0x7f84e403ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 --2- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f84e4038870 0x7f84e403ad30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f84f0006fd0 tx=0x7f84f0006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 msgr2=0x7f84fc19cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 --2- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc19cc20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f84ec011670 tx=0x7f84ec011a30 comp rx=0 tx=0).stop 2023-12-12T09:50:30.768 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 shutdown_connections 2023-12-12T09:50:30.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 --2- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f84e4038870 0x7f84e403ad30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:30.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 --2- 172.21.15.42:0/2510345322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84fc1055d0 0x7f84fc19cc20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:30.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 >> 172.21.15.42:0/2510345322 conn(0x7f84fc1011d0 msgr2=0x7f84fc193a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:30.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 shutdown_connections 2023-12-12T09:50:30.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:30.276+0000 7f8501f9a700 1 -- 172.21.15.42:0/2510345322 wait complete. 2023-12-12T09:50:30.769 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial admin user... 2023-12-12T09:50:31.027 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:30 smithi042 ceph-mon[98894]: mgrmap e12: x(active, since 2s) 2023-12-12T09:50:31.027 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:30 smithi042 ceph-mon[98894]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:31.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:31.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:32.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:32.516 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$zGaWDwvBteAXjy52oWv1ketfT.Eei.nBhDa0rnPxuxkRLtP50/yXu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1702374631, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-12T09:50:32.516 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.325+0000 7fd7e13b3700 1 Processor -- start 2023-12-12T09:50:32.516 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.325+0000 7fd7e13b3700 1 -- start start 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.325+0000 7fd7e13b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc105a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.325+0000 7fd7e13b3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc07adc0 con 0x7fd7dc105630 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.325+0000 7fd7daffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc105a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.325+0000 7fd7daffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc105a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60616/0 (socket says 172.21.15.42:60616) 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.325+0000 7fd7daffd700 1 -- 172.21.15.42:0/3525812911 learned_addr learned my addr 172.21.15.42:0/3525812911 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.325+0000 7fd7daffd700 1 -- 172.21.15.42:0/3525812911 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc07af00 con 0x7fd7dc105630 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.326+0000 7fd7daffd700 1 --2- 172.21.15.42:0/3525812911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc105a50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0098a0 tx=0x7fd7cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804d6f12363aa05a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.326+0000 7fd7d9ffb700 1 -- 172.21.15.42:0/3525812911 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7cc004160 con 0x7fd7dc105630 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.326+0000 7fd7d9ffb700 1 -- 172.21.15.42:0/3525812911 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7cc0042c0 con 0x7fd7dc105630 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.326+0000 7fd7d9ffb700 1 -- 172.21.15.42:0/3525812911 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7cc0119b0 con 0x7fd7dc105630 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.326+0000 7fd7e13b3700 1 -- 172.21.15.42:0/3525812911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 msgr2=0x7fd7dc105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.326+0000 7fd7e13b3700 1 --2- 172.21.15.42:0/3525812911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc105a50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0098a0 tx=0x7fd7cc009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:32.517 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.326+0000 7fd7e13b3700 1 -- 172.21.15.42:0/3525812911 shutdown_connections 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.326+0000 7fd7e13b3700 1 --2- 172.21.15.42:0/3525812911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc105a50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.326+0000 7fd7e13b3700 1 -- 172.21.15.42:0/3525812911 >> 172.21.15.42:0/3525812911 conn(0x7fd7dc1011d0 msgr2=0x7fd7dc103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.327+0000 7fd7e13b3700 1 -- 172.21.15.42:0/3525812911 shutdown_connections 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.327+0000 7fd7e13b3700 1 -- 172.21.15.42:0/3525812911 wait complete. 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.327+0000 7fd7e13b3700 1 Processor -- start 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.328+0000 7fd7e13b3700 1 -- start start 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.328+0000 7fd7e13b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.328+0000 7fd7e13b3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc19d150 con 0x7fd7dc105630 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.328+0000 7fd7daffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.328+0000 7fd7daffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc19cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60618/0 (socket says 172.21.15.42:60618) 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.328+0000 7fd7daffd700 1 -- 172.21.15.42:0/604480531 learned_addr learned my addr 172.21.15.42:0/604480531 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:32.518 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.328+0000 7fd7daffd700 1 -- 172.21.15.42:0/604480531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc009550 con 0x7fd7dc105630 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.328+0000 7fd7daffd700 1 --2- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc19cc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc006af0 tx=0x7fd7cc011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.329+0000 7fd7c3fff700 1 -- 172.21.15.42:0/604480531 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7cc003a40 con 0x7fd7dc105630 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.329+0000 7fd7c3fff700 1 -- 172.21.15.42:0/604480531 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7cc003ba0 con 0x7fd7dc105630 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.329+0000 7fd7c3fff700 1 -- 172.21.15.42:0/604480531 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7cc0196c0 con 0x7fd7dc105630 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.329+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc19d350 con 0x7fd7dc105630 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.329+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc19d7f0 con 0x7fd7dc105630 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.330+0000 7fd7c3fff700 1 -- 172.21.15.42:0/604480531 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7fd7cc01aa80 con 0x7fd7dc105630 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.330+0000 7fd7c3fff700 1 --2- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7c4038870 0x7fd7c403ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.330+0000 7fd7c3fff700 1 -- 172.21.15.42:0/604480531 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd7cc04cd20 con 0x7fd7dc105630 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.330+0000 7fd7da7fc700 1 --2- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7c4038870 0x7fd7c403ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.330+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7dc196960 con 0x7fd7dc105630 2023-12-12T09:50:32.519 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.330+0000 7fd7da7fc700 1 --2- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7c4038870 0x7fd7c403ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0006fd0 tx=0x7fd7d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.335+0000 7fd7c3fff700 1 -- 172.21.15.42:0/604480531 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd7cc02d430 con 0x7fd7dc105630 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.744+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd7dc106cb0 con 0x7fd7c4038870 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.991+0000 7fd7c3fff700 1 -- 172.21.15.42:0/604480531 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fd7dc106cb0 con 0x7fd7c4038870 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.993+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7c4038870 msgr2=0x7fd7c403ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.993+0000 7fd7e13b3700 1 --2- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7c4038870 0x7fd7c403ad30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0006fd0 tx=0x7fd7d0006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.993+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 msgr2=0x7fd7dc19cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.993+0000 7fd7e13b3700 1 --2- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc19cc10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc006af0 tx=0x7fd7cc011a10 comp rx=0 tx=0).stop 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.994+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 shutdown_connections 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.994+0000 7fd7e13b3700 1 --2- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7c4038870 0x7fd7c403ad30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.994+0000 7fd7e13b3700 1 --2- 172.21.15.42:0/604480531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7dc105630 0x7fd7dc19cc10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.994+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 >> 172.21.15.42:0/604480531 conn(0x7fd7dc1011d0 msgr2=0x7fd7dc193aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:32.520 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.994+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 shutdown_connections 2023-12-12T09:50:32.521 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:31.994+0000 7fd7e13b3700 1 -- 172.21.15.42:0/604480531 wait complete. 2023-12-12T09:50:32.521 INFO:teuthology.orchestra.run.smithi042.stdout:Fetching dashboard port number... 2023-12-12T09:50:33.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:32 smithi042 ceph-mon[98894]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:33.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:33.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 8443 2023-12-12T09:50:33.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.011+0000 7f823f710700 1 Processor -- start 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.011+0000 7f823f710700 1 -- start start 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.012+0000 7f823f710700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f8238107100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.012+0000 7f823f710700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82381076d0 con 0x7f8238106ce0 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.012+0000 7f823d4ac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f8238107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.012+0000 7f823d4ac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f8238107100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60622/0 (socket says 172.21.15.42:60622) 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.012+0000 7f823d4ac700 1 -- 172.21.15.42:0/889955910 learned_addr learned my addr 172.21.15.42:0/889955910 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.012+0000 7f823d4ac700 1 -- 172.21.15.42:0/889955910 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8238107ee0 con 0x7f8238106ce0 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.012+0000 7f823d4ac700 1 --2- 172.21.15.42:0/889955910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f8238107100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82340098a0 tx=0x7f8234009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc374643b0f2f081 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.012+0000 7f822ffff700 1 -- 172.21.15.42:0/889955910 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8234004160 con 0x7f8238106ce0 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.013+0000 7f822ffff700 1 -- 172.21.15.42:0/889955910 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f82340042c0 con 0x7f8238106ce0 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.013+0000 7f822ffff700 1 -- 172.21.15.42:0/889955910 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82340119b0 con 0x7f8238106ce0 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.013+0000 7f823f710700 1 -- 172.21.15.42:0/889955910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 msgr2=0x7f8238107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.013+0000 7f823f710700 1 --2- 172.21.15.42:0/889955910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f8238107100 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f82340098a0 tx=0x7f8234009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:33.864 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.013+0000 7f823f710700 1 -- 172.21.15.42:0/889955910 shutdown_connections 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.013+0000 7f823f710700 1 --2- 172.21.15.42:0/889955910 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f8238107100 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.013+0000 7f823f710700 1 -- 172.21.15.42:0/889955910 >> 172.21.15.42:0/889955910 conn(0x7f8238102280 msgr2=0x7f82381046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.013+0000 7f823f710700 1 -- 172.21.15.42:0/889955910 shutdown_connections 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.014+0000 7f823f710700 1 -- 172.21.15.42:0/889955910 wait complete. 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.014+0000 7f823f710700 1 Processor -- start 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.014+0000 7f823f710700 1 -- start start 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.014+0000 7f823f710700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f823807ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.014+0000 7f823f710700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823807b250 con 0x7f8238106ce0 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.014+0000 7f823d4ac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f823807ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.014+0000 7f823d4ac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f823807ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60624/0 (socket says 172.21.15.42:60624) 2023-12-12T09:50:33.865 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.014+0000 7f823d4ac700 1 -- 172.21.15.42:0/1200165857 learned_addr learned my addr 172.21.15.42:0/1200165857 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.015+0000 7f823d4ac700 1 -- 172.21.15.42:0/1200165857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8234009550 con 0x7f8238106ce0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.015+0000 7f823d4ac700 1 --2- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f823807ccb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8234003e60 tx=0x7f8234011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.015+0000 7f822e7fc700 1 -- 172.21.15.42:0/1200165857 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8234003a60 con 0x7f8238106ce0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.015+0000 7f822e7fc700 1 -- 172.21.15.42:0/1200165857 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8234003bc0 con 0x7f8238106ce0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.015+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f823807b450 con 0x7f8238106ce0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.015+0000 7f822e7fc700 1 -- 172.21.15.42:0/1200165857 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82340196c0 con 0x7f8238106ce0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.015+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823807b8f0 con 0x7f8238106ce0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.016+0000 7f822e7fc700 1 -- 172.21.15.42:0/1200165857 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) v1 ==== 45593+0+0 (secure 0 0 0) 0x7f8234019980 con 0x7f8238106ce0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.016+0000 7f822e7fc700 1 --2- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8224038850 0x7f822403ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.016+0000 7f822e7fc700 1 -- 172.21.15.42:0/1200165857 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f823404dba0 con 0x7f8238106ce0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.016+0000 7f823ccab700 1 --2- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8224038850 0x7f822403ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:33.866 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.017+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c005320 con 0x7f8238106ce0 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.017+0000 7f823ccab700 1 --2- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8224038850 0x7f822403ad10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8228006fd0 tx=0x7f8228006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.022+0000 7f822e7fc700 1 -- 172.21.15.42:0/1200165857 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8234019cd0 con 0x7f8238106ce0 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.385+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f821c005b50 con 0x7f8238106ce0 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.385+0000 7f822e7fc700 1 -- 172.21.15.42:0/1200165857 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f8234010b70 con 0x7f8238106ce0 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8224038850 msgr2=0x7f822403ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 --2- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8224038850 0x7f822403ad10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8228006fd0 tx=0x7f8228006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 msgr2=0x7f823807ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 --2- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f823807ccb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8234003e60 tx=0x7f8234011a30 comp rx=0 tx=0).stop 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 shutdown_connections 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 --2- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8224038850 0x7f822403ad10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 --2- 172.21.15.42:0/1200165857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8238106ce0 0x7f823807ccb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 >> 172.21.15.42:0/1200165857 conn(0x7f8238102280 msgr2=0x7f8238102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 shutdown_connections 2023-12-12T09:50:33.867 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:33.388+0000 7f823f710700 1 -- 172.21.15.42:0/1200165857 wait complete. 2023-12-12T09:50:33.872 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-12T09:50:33.872 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2023-12-12T09:50:33.881 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-12T09:50:33.882 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2023-12-12T09:50:33.882 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2023-12-12T09:50:33.882 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-12T09:50:33.883 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph Dashboard is now available at: 2023-12-12T09:50:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: URL: https://smithi042.front.sepia.ceph.com:8443/ 2023-12-12T09:50:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: User: admin 2023-12-12T09:50:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: Password: l4elpijh72 2023-12-12T09:50:33.883 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:33.884 INFO:teuthology.orchestra.run.smithi042.stdout:Saving cluster configuration to /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config directory 2023-12-12T09:50:33.884 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling autotune for osd_memory_target 2023-12-12T09:50:34.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:34 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1200165857' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-12T09:50:35.134 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.356+0000 7f5c4d51b700 1 Processor -- start 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.356+0000 7f5c4d51b700 1 -- start start 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.356+0000 7f5c4d51b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48106030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.356+0000 7f5c4d51b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c48106600 con 0x7f5c48105c10 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.357+0000 7f5c46ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48106030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.357+0000 7f5c46ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48106030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60632/0 (socket says 172.21.15.42:60632) 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.357+0000 7f5c46ffd700 1 -- 172.21.15.42:0/736957528 learned_addr learned my addr 172.21.15.42:0/736957528 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.357+0000 7f5c46ffd700 1 -- 172.21.15.42:0/736957528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48106e10 con 0x7f5c48105c10 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.357+0000 7f5c46ffd700 1 --2- 172.21.15.42:0/736957528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48106030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c300098a0 tx=0x7f5c30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63eb446446029fa4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.357+0000 7f5c45ffb700 1 -- 172.21.15.42:0/736957528 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c30004160 con 0x7f5c48105c10 2023-12-12T09:50:35.135 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.357+0000 7f5c45ffb700 1 -- 172.21.15.42:0/736957528 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c300042c0 con 0x7f5c48105c10 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.357+0000 7f5c45ffb700 1 -- 172.21.15.42:0/736957528 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c300119b0 con 0x7f5c48105c10 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.358+0000 7f5c4d51b700 1 -- 172.21.15.42:0/736957528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 msgr2=0x7f5c48106030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.358+0000 7f5c4d51b700 1 --2- 172.21.15.42:0/736957528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48106030 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5c300098a0 tx=0x7f5c30009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.358+0000 7f5c4d51b700 1 -- 172.21.15.42:0/736957528 shutdown_connections 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.358+0000 7f5c4d51b700 1 --2- 172.21.15.42:0/736957528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48106030 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.358+0000 7f5c4d51b700 1 -- 172.21.15.42:0/736957528 >> 172.21.15.42:0/736957528 conn(0x7f5c481011d0 msgr2=0x7f5c481035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.358+0000 7f5c4d51b700 1 -- 172.21.15.42:0/736957528 shutdown_connections 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.358+0000 7f5c4d51b700 1 -- 172.21.15.42:0/736957528 wait complete. 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.359+0000 7f5c4d51b700 1 Processor -- start 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.359+0000 7f5c4d51b700 1 -- start start 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.359+0000 7f5c4d51b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48190f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.359+0000 7f5c4d51b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4818f4b0 con 0x7f5c48105c10 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.359+0000 7f5c46ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48190f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:35.136 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.359+0000 7f5c46ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48190f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60634/0 (socket says 172.21.15.42:60634) 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.359+0000 7f5c46ffd700 1 -- 172.21.15.42:0/3983288585 learned_addr learned my addr 172.21.15.42:0/3983288585 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.359+0000 7f5c46ffd700 1 -- 172.21.15.42:0/3983288585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c30009550 con 0x7f5c48105c10 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.359+0000 7f5c46ffd700 1 --2- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48190f10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c30006af0 tx=0x7f5c30011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.360+0000 7f5c3ffff700 1 -- 172.21.15.42:0/3983288585 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c30003a40 con 0x7f5c48105c10 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.360+0000 7f5c3ffff700 1 -- 172.21.15.42:0/3983288585 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c30003ba0 con 0x7f5c48105c10 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.360+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4818f6b0 con 0x7f5c48105c10 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.360+0000 7f5c3ffff700 1 -- 172.21.15.42:0/3983288585 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c30019690 con 0x7f5c48105c10 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.360+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4818fb50 con 0x7f5c48105c10 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.361+0000 7f5c3ffff700 1 -- 172.21.15.42:0/3983288585 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f5c3001aa60 con 0x7f5c48105c10 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.361+0000 7f5c3ffff700 1 --2- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c34038910 0x7f5c3403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.361+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c48194cb0 con 0x7f5c48105c10 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.361+0000 7f5c3ffff700 1 -- 172.21.15.42:0/3983288585 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5c3004cc50 con 0x7f5c48105c10 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.361+0000 7f5c467fc700 1 --2- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c34038910 0x7f5c3403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.362+0000 7f5c467fc700 1 --2- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c34038910 0x7f5c3403add0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c38006fd0 tx=0x7f5c38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:35.137 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.366+0000 7f5c3ffff700 1 -- 172.21.15.42:0/3983288585 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5c3002d430 con 0x7f5c48105c10 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.727+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f5c4810a2c0 con 0x7f5c48105c10 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.728+0000 7f5c3ffff700 1 -- 172.21.15.42:0/3983288585 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f5c30023390 con 0x7f5c48105c10 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c34038910 msgr2=0x7f5c3403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 --2- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c34038910 0x7f5c3403add0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5c38006fd0 tx=0x7f5c38006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 msgr2=0x7f5c48190f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 --2- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48190f10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5c30006af0 tx=0x7f5c30011a10 comp rx=0 tx=0).stop 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 shutdown_connections 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 --2- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c34038910 0x7f5c3403add0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 --2- 172.21.15.42:0/3983288585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48105c10 0x7f5c48190f10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 >> 172.21.15.42:0/3983288585 conn(0x7f5c481011d0 msgr2=0x7f5c48102070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 shutdown_connections 2023-12-12T09:50:35.138 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:34.730+0000 7f5c4d51b700 1 -- 172.21.15.42:0/3983288585 wait complete. 2023-12-12T09:50:35.331 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:35 smithi042 ceph-mon[98894]: mgrmap e13: x(active, since 6s) 2023-12-12T09:50:36.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.584+0000 7fbd4ebd9700 1 Processor -- start 2023-12-12T09:50:36.582 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.584+0000 7fbd4ebd9700 1 -- start start 2023-12-12T09:50:36.582 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.585+0000 7fbd4ebd9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd48107110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:36.582 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.585+0000 7fbd4ebd9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd481076e0 con 0x7fbd48106cf0 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.585+0000 7fbd4c975700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd48107110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.585+0000 7fbd4c975700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd48107110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60638/0 (socket says 172.21.15.42:60638) 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.585+0000 7fbd4c975700 1 -- 172.21.15.42:0/1996735417 learned_addr learned my addr 172.21.15.42:0/1996735417 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.585+0000 7fbd4c975700 1 -- 172.21.15.42:0/1996735417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd48107ef0 con 0x7fbd48106cf0 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd4c975700 1 --2- 172.21.15.42:0/1996735417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd48107110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0098a0 tx=0x7fbd3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fb5e385be33fc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd477fe700 1 -- 172.21.15.42:0/1996735417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c004160 con 0x7fbd48106cf0 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd477fe700 1 -- 172.21.15.42:0/1996735417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd3c0042c0 con 0x7fbd48106cf0 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd477fe700 1 -- 172.21.15.42:0/1996735417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c0119b0 con 0x7fbd48106cf0 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1996735417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 msgr2=0x7fbd48107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd4ebd9700 1 --2- 172.21.15.42:0/1996735417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd48107110 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c0098a0 tx=0x7fbd3c009bb0 comp rx=0 tx=0).stop 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1996735417 shutdown_connections 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd4ebd9700 1 --2- 172.21.15.42:0/1996735417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd48107110 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1996735417 >> 172.21.15.42:0/1996735417 conn(0x7fbd481022d0 msgr2=0x7fbd481046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1996735417 shutdown_connections 2023-12-12T09:50:36.583 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.586+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1996735417 wait complete. 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.587+0000 7fbd4ebd9700 1 Processor -- start 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.587+0000 7fbd4ebd9700 1 -- start start 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.587+0000 7fbd4ebd9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd4807ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.587+0000 7fbd4ebd9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4807b250 con 0x7fbd48106cf0 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.587+0000 7fbd4c975700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd4807ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.587+0000 7fbd4c975700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd4807ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60640/0 (socket says 172.21.15.42:60640) 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.587+0000 7fbd4c975700 1 -- 172.21.15.42:0/1694064999 learned_addr learned my addr 172.21.15.42:0/1694064999 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.588+0000 7fbd4c975700 1 -- 172.21.15.42:0/1694064999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c009550 con 0x7fbd48106cf0 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.588+0000 7fbd4c975700 1 --2- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd4807ccb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c003e60 tx=0x7fbd3c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.588+0000 7fbd45ffb700 1 -- 172.21.15.42:0/1694064999 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c003a60 con 0x7fbd48106cf0 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.588+0000 7fbd45ffb700 1 -- 172.21.15.42:0/1694064999 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd3c003bc0 con 0x7fbd48106cf0 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.588+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4807b450 con 0x7fbd48106cf0 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.588+0000 7fbd45ffb700 1 -- 172.21.15.42:0/1694064999 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbd3c0196c0 con 0x7fbd48106cf0 2023-12-12T09:50:36.584 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.588+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4807b8f0 con 0x7fbd48106cf0 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.589+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd48067940 con 0x7fbd48106cf0 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.589+0000 7fbd45ffb700 1 -- 172.21.15.42:0/1694064999 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fbd3c01ab60 con 0x7fbd48106cf0 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.590+0000 7fbd45ffb700 1 --2- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbd380388f0 0x7fbd3803adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.590+0000 7fbd45ffb700 1 -- 172.21.15.42:0/1694064999 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbd3c04cc80 con 0x7fbd48106cf0 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.590+0000 7fbd47fff700 1 --2- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbd380388f0 0x7fbd3803adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.590+0000 7fbd47fff700 1 --2- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbd380388f0 0x7fbd3803adb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd34006fd0 tx=0x7fbd34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:35.594+0000 7fbd45ffb700 1 -- 172.21.15.42:0/1694064999 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbd3c010740 con 0x7fbd48106cf0 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.178+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fbd4810b120 con 0x7fbd48106cf0 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.180+0000 7fbd45ffb700 1 -- 172.21.15.42:0/1694064999 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fbd3c0281c0 con 0x7fbd48106cf0 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbd380388f0 msgr2=0x7fbd3803adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 --2- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbd380388f0 0x7fbd3803adb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbd34006fd0 tx=0x7fbd34006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 msgr2=0x7fbd4807ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 --2- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd4807ccb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c003e60 tx=0x7fbd3c011a30 comp rx=0 tx=0).stop 2023-12-12T09:50:36.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 shutdown_connections 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 --2- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbd380388f0 0x7fbd3803adb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 --2- 172.21.15.42:0/1694064999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbd48106cf0 0x7fbd4807ccb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 >> 172.21.15.42:0/1694064999 conn(0x7fbd481022d0 msgr2=0x7fbd48102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 shutdown_connections 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2023-12-12T09:50:36.183+0000 7fbd4ebd9700 1 -- 172.21.15.42:0/1694064999 wait complete. 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout:Or, if you are only running a single cluster on this host: 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-12-12T09:50:36.586 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:36.587 INFO:teuthology.orchestra.run.smithi042.stdout:Please consider enabling telemetry to help improve Ceph: 2023-12-12T09:50:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: ceph telemetry on 2023-12-12T09:50:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:36.587 INFO:teuthology.orchestra.run.smithi042.stdout:For more information see: 2023-12-12T09:50:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-12-12T09:50:36.587 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:36.587 INFO:teuthology.orchestra.run.smithi042.stdout:Bootstrap complete. 2023-12-12T09:50:36.625 INFO:tasks.cephadm:Fetching config... 2023-12-12T09:50:36.626 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:50:36.626 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-12T09:50:36.643 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-12T09:50:36.643 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:50:36.643 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-12T09:50:36.702 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-12T09:50:36.703 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:50:36.703 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/keyring of=/dev/stdout 2023-12-12T09:50:36.770 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-12T09:50:36.771 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:50:36.771 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-12T09:50:36.827 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-12T09:50:36.828 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDt4fi1xpyNYb3S4s0HRhz63+Pa8ivzFs+59IgK+zylN60YKGI5JVKjA+b19YJrRahc7k+ejtWYV3ReCCd6LhQ7rvuSxVEKAMBLxfGXpGmT4NeiCb0T0hoi+D7MUedjEYFdqIudjtyC++4KC2Yt7BQ4yBsgajVWhalzT776devXwujy4oBVjF1ffVTt+JZFHCPhTfbaHEI/CLXgaCqVS7Owtg2A3jJOa65drkTTU98+AI0feBIvn2BkS5DQxdVcKI8EuZzIkOhrv2SfXFUDUEAlO0TrrEHpwZXexAb4SSixwKu+uBJaZ7iyawQFcb+p8jq+w0lFiqFiL+Qw4COl/FPSE+e8pYBvDx4GSjIuyHjtFZV4TVdQuFgMGDUk0EL1l704VmrGv/9x3wxUYm71/Jhc66uXPsJmU/XRiP4QaYhFb1ETKKReR8TIDiogjk56CmlMSoPHNchMoiic+w5MOa6Zp5NtRP6RV9J5QXDXNJqlnO1Bz4XADhryCj36mMPmiPs= ceph-968ccaec-98d3-11ee-95a3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-12T09:50:36.916 INFO:teuthology.orchestra.run.smithi042.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDt4fi1xpyNYb3S4s0HRhz63+Pa8ivzFs+59IgK+zylN60YKGI5JVKjA+b19YJrRahc7k+ejtWYV3ReCCd6LhQ7rvuSxVEKAMBLxfGXpGmT4NeiCb0T0hoi+D7MUedjEYFdqIudjtyC++4KC2Yt7BQ4yBsgajVWhalzT776devXwujy4oBVjF1ffVTt+JZFHCPhTfbaHEI/CLXgaCqVS7Owtg2A3jJOa65drkTTU98+AI0feBIvn2BkS5DQxdVcKI8EuZzIkOhrv2SfXFUDUEAlO0TrrEHpwZXexAb4SSixwKu+uBJaZ7iyawQFcb+p8jq+w0lFiqFiL+Qw4COl/FPSE+e8pYBvDx4GSjIuyHjtFZV4TVdQuFgMGDUk0EL1l704VmrGv/9x3wxUYm71/Jhc66uXPsJmU/XRiP4QaYhFb1ETKKReR8TIDiogjk56CmlMSoPHNchMoiic+w5MOa6Zp5NtRP6RV9J5QXDXNJqlnO1Bz4XADhryCj36mMPmiPs= ceph-968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:50:36.928 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDt4fi1xpyNYb3S4s0HRhz63+Pa8ivzFs+59IgK+zylN60YKGI5JVKjA+b19YJrRahc7k+ejtWYV3ReCCd6LhQ7rvuSxVEKAMBLxfGXpGmT4NeiCb0T0hoi+D7MUedjEYFdqIudjtyC++4KC2Yt7BQ4yBsgajVWhalzT776devXwujy4oBVjF1ffVTt+JZFHCPhTfbaHEI/CLXgaCqVS7Owtg2A3jJOa65drkTTU98+AI0feBIvn2BkS5DQxdVcKI8EuZzIkOhrv2SfXFUDUEAlO0TrrEHpwZXexAb4SSixwKu+uBJaZ7iyawQFcb+p8jq+w0lFiqFiL+Qw4COl/FPSE+e8pYBvDx4GSjIuyHjtFZV4TVdQuFgMGDUk0EL1l704VmrGv/9x3wxUYm71/Jhc66uXPsJmU/XRiP4QaYhFb1ETKKReR8TIDiogjk56CmlMSoPHNchMoiic+w5MOa6Zp5NtRP6RV9J5QXDXNJqlnO1Bz4XADhryCj36mMPmiPs= ceph-968ccaec-98d3-11ee-95a3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-12T09:50:36.976 INFO:teuthology.orchestra.run.smithi111.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDt4fi1xpyNYb3S4s0HRhz63+Pa8ivzFs+59IgK+zylN60YKGI5JVKjA+b19YJrRahc7k+ejtWYV3ReCCd6LhQ7rvuSxVEKAMBLxfGXpGmT4NeiCb0T0hoi+D7MUedjEYFdqIudjtyC++4KC2Yt7BQ4yBsgajVWhalzT776devXwujy4oBVjF1ffVTt+JZFHCPhTfbaHEI/CLXgaCqVS7Owtg2A3jJOa65drkTTU98+AI0feBIvn2BkS5DQxdVcKI8EuZzIkOhrv2SfXFUDUEAlO0TrrEHpwZXexAb4SSixwKu+uBJaZ7iyawQFcb+p8jq+w0lFiqFiL+Qw4COl/FPSE+e8pYBvDx4GSjIuyHjtFZV4TVdQuFgMGDUk0EL1l704VmrGv/9x3wxUYm71/Jhc66uXPsJmU/XRiP4QaYhFb1ETKKReR8TIDiogjk56CmlMSoPHNchMoiic+w5MOa6Zp5NtRP6RV9J5QXDXNJqlnO1Bz4XADhryCj36mMPmiPs= ceph-968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:50:36.989 DEBUG:teuthology.orchestra.run.smithi173:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDt4fi1xpyNYb3S4s0HRhz63+Pa8ivzFs+59IgK+zylN60YKGI5JVKjA+b19YJrRahc7k+ejtWYV3ReCCd6LhQ7rvuSxVEKAMBLxfGXpGmT4NeiCb0T0hoi+D7MUedjEYFdqIudjtyC++4KC2Yt7BQ4yBsgajVWhalzT776devXwujy4oBVjF1ffVTt+JZFHCPhTfbaHEI/CLXgaCqVS7Owtg2A3jJOa65drkTTU98+AI0feBIvn2BkS5DQxdVcKI8EuZzIkOhrv2SfXFUDUEAlO0TrrEHpwZXexAb4SSixwKu+uBJaZ7iyawQFcb+p8jq+w0lFiqFiL+Qw4COl/FPSE+e8pYBvDx4GSjIuyHjtFZV4TVdQuFgMGDUk0EL1l704VmrGv/9x3wxUYm71/Jhc66uXPsJmU/XRiP4QaYhFb1ETKKReR8TIDiogjk56CmlMSoPHNchMoiic+w5MOa6Zp5NtRP6RV9J5QXDXNJqlnO1Bz4XADhryCj36mMPmiPs= ceph-968ccaec-98d3-11ee-95a3-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-12T09:50:37.036 INFO:teuthology.orchestra.run.smithi173.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDt4fi1xpyNYb3S4s0HRhz63+Pa8ivzFs+59IgK+zylN60YKGI5JVKjA+b19YJrRahc7k+ejtWYV3ReCCd6LhQ7rvuSxVEKAMBLxfGXpGmT4NeiCb0T0hoi+D7MUedjEYFdqIudjtyC++4KC2Yt7BQ4yBsgajVWhalzT776devXwujy4oBVjF1ffVTt+JZFHCPhTfbaHEI/CLXgaCqVS7Owtg2A3jJOa65drkTTU98+AI0feBIvn2BkS5DQxdVcKI8EuZzIkOhrv2SfXFUDUEAlO0TrrEHpwZXexAb4SSixwKu+uBJaZ7iyawQFcb+p8jq+w0lFiqFiL+Qw4COl/FPSE+e8pYBvDx4GSjIuyHjtFZV4TVdQuFgMGDUk0EL1l704VmrGv/9x3wxUYm71/Jhc66uXPsJmU/XRiP4QaYhFb1ETKKReR8TIDiogjk56CmlMSoPHNchMoiic+w5MOa6Zp5NtRP6RV9J5QXDXNJqlnO1Bz4XADhryCj36mMPmiPs= ceph-968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:50:37.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-12T09:50:37.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:37 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1694064999' entity='client.admin' 2023-12-12T09:50:37.653 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.284+0000 7f3f086ff700 1 -- 172.21.15.42:0/3544314555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 msgr2=0x7f3f00100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.284+0000 7f3f086ff700 1 --2- 172.21.15.42:0/3544314555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 0x7f3f00100280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000b0a0 tx=0x7f3ef000b3b0 comp rx=0 tx=0).stop 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.284+0000 7f3f086ff700 1 -- 172.21.15.42:0/3544314555 shutdown_connections 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.284+0000 7f3f086ff700 1 --2- 172.21.15.42:0/3544314555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 0x7f3f00100280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.284+0000 7f3f086ff700 1 -- 172.21.15.42:0/3544314555 >> 172.21.15.42:0/3544314555 conn(0x7f3f000fb400 msgr2=0x7f3f000fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.284+0000 7f3f086ff700 1 -- 172.21.15.42:0/3544314555 shutdown_connections 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.284+0000 7f3f086ff700 1 -- 172.21.15.42:0/3544314555 wait complete. 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.285+0000 7f3f086ff700 1 Processor -- start 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.285+0000 7f3f086ff700 1 -- start start 2023-12-12T09:50:39.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.285+0000 7f3f086ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 0x7f3f001952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.285+0000 7f3f086ff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f001957e0 con 0x7f3f000ffe60 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.285+0000 7f3f0649b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 0x7f3f001952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.285+0000 7f3f0649b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 0x7f3f001952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60646/0 (socket says 172.21.15.42:60646) 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.285+0000 7f3f0649b700 1 -- 172.21.15.42:0/2580492946 learned_addr learned my addr 172.21.15.42:0/2580492946 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.286+0000 7f3f0649b700 1 -- 172.21.15.42:0/2580492946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef0009d00 con 0x7f3f000ffe60 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.286+0000 7f3f0649b700 1 --2- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 0x7f3f001952a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0015040 tx=0x7f3ef000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.286+0000 7f3eff7fe700 1 -- 172.21.15.42:0/2580492946 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef000e040 con 0x7f3f000ffe60 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.286+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f001959e0 con 0x7f3f000ffe60 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.286+0000 7f3eff7fe700 1 -- 172.21.15.42:0/2580492946 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef0003d40 con 0x7f3f000ffe60 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.286+0000 7f3eff7fe700 1 -- 172.21.15.42:0/2580492946 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ef001d720 con 0x7f3f000ffe60 2023-12-12T09:50:39.289 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.286+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f00195e80 con 0x7f3f000ffe60 2023-12-12T09:50:39.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.287+0000 7f3eff7fe700 1 -- 172.21.15.42:0/2580492946 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f3ef0019070 con 0x7f3f000ffe60 2023-12-12T09:50:39.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.287+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0018eb00 con 0x7f3f000ffe60 2023-12-12T09:50:39.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.287+0000 7f3eff7fe700 1 --2- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee80388c0 0x7f3ee803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:39.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.287+0000 7f3eff7fe700 1 -- 172.21.15.42:0/2580492946 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3ef004caf0 con 0x7f3f000ffe60 2023-12-12T09:50:39.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.287+0000 7f3f05c9a700 1 --2- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee80388c0 0x7f3ee803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:39.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.288+0000 7f3f05c9a700 1 --2- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee80388c0 0x7f3ee803ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4006fd0 tx=0x7f3ef4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.295+0000 7f3eff7fe700 1 -- 172.21.15.42:0/2580492946 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3ef001a380 con 0x7f3f000ffe60 2023-12-12T09:50:39.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.497+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f3f00062c00 con 0x7f3f000ffe60 2023-12-12T09:50:39.502 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.500+0000 7f3eff7fe700 1 -- 172.21.15.42:0/2580492946 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3f00062c00 con 0x7f3f000ffe60 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee80388c0 msgr2=0x7f3ee803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 --2- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee80388c0 0x7f3ee803ad80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4006fd0 tx=0x7f3ef4006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 msgr2=0x7f3f001952a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 --2- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 0x7f3f001952a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0015040 tx=0x7f3ef000bd40 comp rx=0 tx=0).stop 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 shutdown_connections 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 --2- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee80388c0 0x7f3ee803ad80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 --2- 172.21.15.42:0/2580492946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f000ffe60 0x7f3f001952a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 >> 172.21.15.42:0/2580492946 conn(0x7f3f000fb400 msgr2=0x7f3f000fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 shutdown_connections 2023-12-12T09:50:39.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:39.504+0000 7f3f086ff700 1 -- 172.21.15.42:0/2580492946 wait complete. 2023-12-12T09:50:39.880 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:39.880 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:39.880 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi042", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:50:39.880 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:39.880 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:50:39.880 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:39.881 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-12-12T09:50:39.881 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-12-12T09:50:40.626 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:50:40.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:40 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2580492946' entity='client.admin' 2023-12-12T09:50:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:40 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:50:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:40 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.847+0000 7f796aec2700 1 -- 172.21.15.42:0/40877908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 msgr2=0x7f7964101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.847+0000 7f796aec2700 1 --2- 172.21.15.42:0/40877908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 0x7f7964101340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7958009910 tx=0x7f7958009c20 comp rx=0 tx=0).stop 2023-12-12T09:50:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.847+0000 7f796aec2700 1 -- 172.21.15.42:0/40877908 shutdown_connections 2023-12-12T09:50:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.847+0000 7f796aec2700 1 --2- 172.21.15.42:0/40877908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 0x7f7964101340 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.847+0000 7f796aec2700 1 -- 172.21.15.42:0/40877908 >> 172.21.15.42:0/40877908 conn(0x7f79640fc4f0 msgr2=0x7f79640fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.847+0000 7f796aec2700 1 -- 172.21.15.42:0/40877908 shutdown_connections 2023-12-12T09:50:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.847+0000 7f796aec2700 1 -- 172.21.15.42:0/40877908 wait complete. 2023-12-12T09:50:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.848+0000 7f796aec2700 1 Processor -- start 2023-12-12T09:50:41.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.848+0000 7f796aec2700 1 -- start start 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.848+0000 7f796aec2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 0x7f7964075240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.848+0000 7f796aec2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7964075780 con 0x7f7964100f00 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.848+0000 7f7968c5e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 0x7f7964075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.848+0000 7f7968c5e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 0x7f7964075240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60652/0 (socket says 172.21.15.42:60652) 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.848+0000 7f7968c5e700 1 -- 172.21.15.42:0/572356834 learned_addr learned my addr 172.21.15.42:0/572356834 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.849+0000 7f7968c5e700 1 -- 172.21.15.42:0/572356834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79580095f0 con 0x7f7964100f00 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.849+0000 7f7968c5e700 1 --2- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 0x7f7964075240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7958004f70 tx=0x7f7958004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.849+0000 7f7961ffb700 1 -- 172.21.15.42:0/572356834 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f795801d070 con 0x7f7964100f00 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.849+0000 7f7961ffb700 1 -- 172.21.15.42:0/572356834 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f795800be20 con 0x7f7964100f00 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.849+0000 7f7961ffb700 1 -- 172.21.15.42:0/572356834 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7958022df0 con 0x7f7964100f00 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.849+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79640739c0 con 0x7f7964100f00 2023-12-12T09:50:41.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.849+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7964073ec0 con 0x7f7964100f00 2023-12-12T09:50:41.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.850+0000 7f7961ffb700 1 -- 172.21.15.42:0/572356834 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f795800b960 con 0x7f7964100f00 2023-12-12T09:50:41.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.850+0000 7f7961ffb700 1 --2- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7954038910 0x7f795403add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:41.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.850+0000 7f7961ffb700 1 -- 172.21.15.42:0/572356834 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f795804ca10 con 0x7f7964100f00 2023-12-12T09:50:41.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.850+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79641074a0 con 0x7f7964100f00 2023-12-12T09:50:41.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.850+0000 7f7963fff700 1 --2- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7954038910 0x7f795403add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:41.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.851+0000 7f7963fff700 1 --2- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7954038910 0x7f795403add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7950006fd0 tx=0x7f7950006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:41.856 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:41.855+0000 7f7961ffb700 1 -- 172.21.15.42:0/572356834 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7958027030 con 0x7f7964100f00 2023-12-12T09:50:42.059 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.058+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f7964074100 con 0x7f7954038910 2023-12-12T09:50:42.063 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.061+0000 7f7961ffb700 1 -- 172.21.15.42:0/572356834 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7964074100 con 0x7f7954038910 2023-12-12T09:50:42.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7954038910 msgr2=0x7f795403add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:42.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 --2- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7954038910 0x7f795403add0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7950006fd0 tx=0x7f7950006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:42.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 msgr2=0x7f7964075240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:42.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 --2- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 0x7f7964075240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7958004f70 tx=0x7f7958004fa0 comp rx=0 tx=0).stop 2023-12-12T09:50:42.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 shutdown_connections 2023-12-12T09:50:42.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 --2- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7954038910 0x7f795403add0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:42.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 --2- 172.21.15.42:0/572356834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7964100f00 0x7f7964075240 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:42.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 >> 172.21.15.42:0/572356834 conn(0x7f79640fc4f0 msgr2=0x7f79640fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:42.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 shutdown_connections 2023-12-12T09:50:42.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:42.064+0000 7f796aec2700 1 -- 172.21.15.42:0/572356834 wait complete. 2023-12-12T09:50:42.487 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi111 2023-12-12T09:50:42.487 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:50:42.487 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.conf 2023-12-12T09:50:42.507 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:50:42.507 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:50:42.567 INFO:tasks.cephadm:Adding host smithi111 to orchestrator... 2023-12-12T09:50:42.568 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch host add smithi111 2023-12-12T09:50:43.180 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:50:43.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:43 smithi042 ceph-mon[98894]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:50:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:50:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:50:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:43 smithi042 ceph-mon[98894]: Updating smithi042:/etc/ceph/ceph.conf 2023-12-12T09:50:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:43 smithi042 ceph-mon[98894]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:50:44.390 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.387+0000 7f0953fc9700 1 -- 172.21.15.42:0/2489423350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 msgr2=0x7f094c068710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.387+0000 7f0953fc9700 1 --2- 172.21.15.42:0/2489423350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 0x7f094c068710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f093c009910 tx=0x7f093c009c20 comp rx=0 tx=0).stop 2023-12-12T09:50:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.387+0000 7f0953fc9700 1 -- 172.21.15.42:0/2489423350 shutdown_connections 2023-12-12T09:50:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.387+0000 7f0953fc9700 1 --2- 172.21.15.42:0/2489423350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 0x7f094c068710 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.387+0000 7f0953fc9700 1 -- 172.21.15.42:0/2489423350 >> 172.21.15.42:0/2489423350 conn(0x7f094c075930 msgr2=0x7f094c075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.387+0000 7f0953fc9700 1 -- 172.21.15.42:0/2489423350 shutdown_connections 2023-12-12T09:50:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.387+0000 7f0953fc9700 1 -- 172.21.15.42:0/2489423350 wait complete. 2023-12-12T09:50:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.388+0000 7f0953fc9700 1 Processor -- start 2023-12-12T09:50:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.388+0000 7f0953fc9700 1 -- start start 2023-12-12T09:50:44.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.388+0000 7f0953fc9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 0x7f094c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.388+0000 7f0953fc9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c19d340 con 0x7f094c105e20 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.388+0000 7f0951d65700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 0x7f094c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.388+0000 7f0951d65700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 0x7f094c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60658/0 (socket says 172.21.15.42:60658) 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.388+0000 7f0951d65700 1 -- 172.21.15.42:0/407534427 learned_addr learned my addr 172.21.15.42:0/407534427 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.388+0000 7f0951d65700 1 -- 172.21.15.42:0/407534427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0095f0 con 0x7f094c105e20 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.389+0000 7f0951d65700 1 --2- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 0x7f094c19ce00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f093c006010 tx=0x7f093c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.389+0000 7f0942ffd700 1 -- 172.21.15.42:0/407534427 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f093c01d070 con 0x7f094c105e20 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.389+0000 7f0942ffd700 1 -- 172.21.15.42:0/407534427 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f093c00bc10 con 0x7f094c105e20 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.389+0000 7f0942ffd700 1 -- 172.21.15.42:0/407534427 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f093c0038a0 con 0x7f094c105e20 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.389+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094c19d540 con 0x7f094c105e20 2023-12-12T09:50:44.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.389+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c19d960 con 0x7f094c105e20 2023-12-12T09:50:44.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.390+0000 7f0942ffd700 1 -- 172.21.15.42:0/407534427 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f093c00b6f0 con 0x7f094c105e20 2023-12-12T09:50:44.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.390+0000 7f0942ffd700 1 --2- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0938038910 0x7f093803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:44.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.390+0000 7f0942ffd700 1 -- 172.21.15.42:0/407534427 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f093c04c4d0 con 0x7f094c105e20 2023-12-12T09:50:44.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.390+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094c10fb60 con 0x7f094c105e20 2023-12-12T09:50:44.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.390+0000 7f0951564700 1 --2- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0938038910 0x7f093803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:44.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.390+0000 7f0951564700 1 --2- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0938038910 0x7f093803add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0948006fd0 tx=0x7f0948006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:44.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.395+0000 7f0942ffd700 1 -- 172.21.15.42:0/407534427 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f093c026020 con 0x7f094c105e20 2023-12-12T09:50:44.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:44 smithi042 ceph-mon[98894]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:50:44.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:44 smithi042 ceph-mon[98894]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:50:44.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:44 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:44.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:44 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:44.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:44 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:44.601 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:44.600+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}) v1 -- 0x7f094c19e180 con 0x7f0938038910 2023-12-12T09:50:45.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:45 smithi042 ceph-mon[98894]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi111", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:46.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:46 smithi042 ceph-mon[98894]: Deploying cephadm binary to smithi111 2023-12-12T09:50:49.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:49 smithi042 ceph-mon[98894]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:50:49.888 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.886+0000 7f0942ffd700 1 -- 172.21.15.42:0/407534427 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f094c19e180 con 0x7f0938038910 2023-12-12T09:50:49.889 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi111' with addr '172.21.15.111' 2023-12-12T09:50:49.890 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0938038910 msgr2=0x7f093803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 --2- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0938038910 0x7f093803add0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0948006fd0 tx=0x7f0948006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 msgr2=0x7f094c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 --2- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 0x7f094c19ce00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f093c006010 tx=0x7f093c004ff0 comp rx=0 tx=0).stop 2023-12-12T09:50:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 shutdown_connections 2023-12-12T09:50:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 --2- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0938038910 0x7f093803add0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 --2- 172.21.15.42:0/407534427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f094c105e20 0x7f094c19ce00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 >> 172.21.15.42:0/407534427 conn(0x7f094c075930 msgr2=0x7f094c100900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 shutdown_connections 2023-12-12T09:50:49.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:49.889+0000 7f0953fc9700 1 -- 172.21.15.42:0/407534427 wait complete. 2023-12-12T09:50:50.349 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch host ls --format=json 2023-12-12T09:50:50.955 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:50:51.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:50 smithi042 ceph-mon[98894]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:50:51.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:50 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:51.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:50 smithi042 ceph-mon[98894]: Added host smithi111 2023-12-12T09:50:51.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:50 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:50:51.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:50 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:52.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.150+0000 7f2546574700 1 -- 172.21.15.42:0/3886386204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 msgr2=0x7f2540106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:52.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.150+0000 7f2546574700 1 --2- 172.21.15.42:0/3886386204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 0x7f2540106250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2534009910 tx=0x7f2534009c20 comp rx=0 tx=0).stop 2023-12-12T09:50:52.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.150+0000 7f2546574700 1 -- 172.21.15.42:0/3886386204 shutdown_connections 2023-12-12T09:50:52.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.150+0000 7f2546574700 1 --2- 172.21.15.42:0/3886386204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 0x7f2540106250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:52.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.150+0000 7f2546574700 1 -- 172.21.15.42:0/3886386204 >> 172.21.15.42:0/3886386204 conn(0x7f2540075930 msgr2=0x7f2540075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:52.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.150+0000 7f2546574700 1 -- 172.21.15.42:0/3886386204 shutdown_connections 2023-12-12T09:50:52.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.150+0000 7f2546574700 1 -- 172.21.15.42:0/3886386204 wait complete. 2023-12-12T09:50:52.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.150+0000 7f2546574700 1 Processor -- start 2023-12-12T09:50:52.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.150+0000 7f2546574700 1 -- start start 2023-12-12T09:50:52.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f2546574700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 0x7f25400fd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f2546574700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25400ff420 con 0x7f2540105e70 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f253ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 0x7f25400fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f253ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 0x7f25400fd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60684/0 (socket says 172.21.15.42:60684) 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f253ffff700 1 -- 172.21.15.42:0/3071628677 learned_addr learned my addr 172.21.15.42:0/3071628677 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f253ffff700 1 -- 172.21.15.42:0/3071628677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25340095f0 con 0x7f2540105e70 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f253ffff700 1 --2- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 0x7f25400fd560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2534006010 tx=0x7f2534004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f253d7fa700 1 -- 172.21.15.42:0/3071628677 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f253401d070 con 0x7f2540105e70 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f253d7fa700 1 -- 172.21.15.42:0/3071628677 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f253400bbb0 con 0x7f2540105e70 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25400fdaa0 con 0x7f2540105e70 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f253d7fa700 1 -- 172.21.15.42:0/3071628677 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2534003810 con 0x7f2540105e70 2023-12-12T09:50:52.154 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.151+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25400fdf20 con 0x7f2540105e70 2023-12-12T09:50:52.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.152+0000 7f253d7fa700 1 -- 172.21.15.42:0/3071628677 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f253400bd20 con 0x7f2540105e70 2023-12-12T09:50:52.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.152+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2540101520 con 0x7f2540105e70 2023-12-12T09:50:52.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.152+0000 7f253d7fa700 1 --2- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2528038910 0x7f252803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:52.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.153+0000 7f253d7fa700 1 -- 172.21.15.42:0/3071628677 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f253404c4f0 con 0x7f2540105e70 2023-12-12T09:50:52.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.153+0000 7f253f7fe700 1 --2- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2528038910 0x7f252803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:52.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.153+0000 7f253f7fe700 1 --2- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2528038910 0x7f252803add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2530006fd0 tx=0x7f2530006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:52.159 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.157+0000 7f253d7fa700 1 -- 172.21.15.42:0/3071628677 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f253402aba0 con 0x7f2540105e70 2023-12-12T09:50:52.365 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.363+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25400feb50 con 0x7f2528038910 2023-12-12T09:50:52.367 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.365+0000 7f253d7fa700 1 -- 172.21.15.42:0/3071628677 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f25400feb50 con 0x7f2528038910 2023-12-12T09:50:52.367 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:50:52.367 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}] 2023-12-12T09:50:52.371 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.369+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2528038910 msgr2=0x7f252803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:52.371 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.369+0000 7f2546574700 1 --2- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2528038910 0x7f252803add0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2530006fd0 tx=0x7f2530006e40 comp rx=0 tx=0).stop 2023-12-12T09:50:52.371 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.369+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 msgr2=0x7f25400fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:52.371 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.369+0000 7f2546574700 1 --2- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 0x7f25400fd560 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2534006010 tx=0x7f2534004ff0 comp rx=0 tx=0).stop 2023-12-12T09:50:52.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.370+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 shutdown_connections 2023-12-12T09:50:52.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.370+0000 7f2546574700 1 --2- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2528038910 0x7f252803add0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:52.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.370+0000 7f2546574700 1 --2- 172.21.15.42:0/3071628677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2540105e70 0x7f25400fd560 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:52.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.370+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 >> 172.21.15.42:0/3071628677 conn(0x7f2540075930 msgr2=0x7f2540068710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:52.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.370+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 shutdown_connections 2023-12-12T09:50:52.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:52.370+0000 7f2546574700 1 -- 172.21.15.42:0/3071628677 wait complete. 2023-12-12T09:50:52.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:52 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:52.821 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi173 2023-12-12T09:50:52.821 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:50:52.821 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.conf 2023-12-12T09:50:52.842 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:50:52.842 DEBUG:teuthology.orchestra.run.smithi173:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:50:52.902 INFO:tasks.cephadm:Adding host smithi173 to orchestrator... 2023-12-12T09:50:52.902 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch host add smithi173 2023-12-12T09:50:53.506 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:50:53.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:53 smithi042 ceph-mon[98894]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:50:53.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:53 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:53.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:53 smithi042 ceph-mon[98894]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:50:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.699+0000 7fdee5332700 1 -- 172.21.15.42:0/641290772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 msgr2=0x7fdee0100420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:50:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.699+0000 7fdee5332700 1 --2- 172.21.15.42:0/641290772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 0x7fdee0100420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fded0009910 tx=0x7fded0009c20 comp rx=0 tx=0).stop 2023-12-12T09:50:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.699+0000 7fdee5332700 1 -- 172.21.15.42:0/641290772 shutdown_connections 2023-12-12T09:50:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.699+0000 7fdee5332700 1 --2- 172.21.15.42:0/641290772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 0x7fdee0100420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:50:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.699+0000 7fdee5332700 1 -- 172.21.15.42:0/641290772 >> 172.21.15.42:0/641290772 conn(0x7fdee00fb8d0 msgr2=0x7fdee00fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:50:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.699+0000 7fdee5332700 1 -- 172.21.15.42:0/641290772 shutdown_connections 2023-12-12T09:50:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.699+0000 7fdee5332700 1 -- 172.21.15.42:0/641290772 wait complete. 2023-12-12T09:50:54.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.699+0000 7fdee5332700 1 Processor -- start 2023-12-12T09:50:54.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdee5332700 1 -- start start 2023-12-12T09:50:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdee5332700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 0x7fdee0195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdee5332700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee006a1c0 con 0x7fdee0100040 2023-12-12T09:50:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdedeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 0x7fdee0195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdedeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 0x7fdee0195430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60690/0 (socket says 172.21.15.42:60690) 2023-12-12T09:50:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdedeffd700 1 -- 172.21.15.42:0/3526503751 learned_addr learned my addr 172.21.15.42:0/3526503751 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:50:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdedeffd700 1 -- 172.21.15.42:0/3526503751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00095f0 con 0x7fdee0100040 2023-12-12T09:50:54.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdedeffd700 1 --2- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 0x7fdee0195430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fded0006010 tx=0x7fded0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdec7fff700 1 -- 172.21.15.42:0/3526503751 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fded001d070 con 0x7fdee0100040 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdec7fff700 1 -- 172.21.15.42:0/3526503751 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fded000bc10 con 0x7fdee0100040 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee0195970 con 0x7fdee0100040 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.700+0000 7fdec7fff700 1 -- 172.21.15.42:0/3526503751 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fded00038a0 con 0x7fdee0100040 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.701+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee0195d90 con 0x7fdee0100040 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.701+0000 7fdec7fff700 1 -- 172.21.15.42:0/3526503751 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fded0003a00 con 0x7fdee0100040 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.702+0000 7fdec7fff700 1 --2- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdec8038910 0x7fdec803add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.702+0000 7fdec7fff700 1 -- 172.21.15.42:0/3526503751 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fded004c5e0 con 0x7fdee0100040 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.702+0000 7fdede7fc700 1 --2- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdec8038910 0x7fdec803add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.702+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee0101520 con 0x7fdee0100040 2023-12-12T09:50:54.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.702+0000 7fdede7fc700 1 --2- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdec8038910 0x7fdec803add0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fded4006fd0 tx=0x7fded4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:50:54.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.707+0000 7fdec7fff700 1 -- 172.21.15.42:0/3526503751 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fded0026070 con 0x7fdee0100040 2023-12-12T09:50:54.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:50:54.909+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}) v1 -- 0x7fdee0069990 con 0x7fdec8038910 2023-12-12T09:50:55.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:55 smithi042 ceph-mon[98894]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:50:55.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:55.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:55.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:55.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:50:55.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:50:55.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:50:55.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:50:56.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:56 smithi042 ceph-mon[98894]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:50:56.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:56 smithi042 ceph-mon[98894]: Updating smithi111:/etc/ceph/ceph.conf 2023-12-12T09:50:56.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:56 smithi042 ceph-mon[98894]: Deploying cephadm binary to smithi173 2023-12-12T09:50:56.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:56 smithi042 ceph-mon[98894]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:50:57.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:57 smithi042 ceph-mon[98894]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:50:57.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:57 smithi042 ceph-mon[98894]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:50:59.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:50:59 smithi042 ceph-mon[98894]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:00.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.278+0000 7fdec7fff700 1 -- 172.21.15.42:0/3526503751 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fdee0069990 con 0x7fdec8038910 2023-12-12T09:51:00.281 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi173' with addr '172.21.15.173' 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdec8038910 msgr2=0x7fdec803add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 --2- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdec8038910 0x7fdec803add0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fded4006fd0 tx=0x7fded4006e40 comp rx=0 tx=0).stop 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 msgr2=0x7fdee0195430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 --2- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 0x7fdee0195430 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fded0006010 tx=0x7fded0004ff0 comp rx=0 tx=0).stop 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 shutdown_connections 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 --2- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdec8038910 0x7fdec803add0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 --2- 172.21.15.42:0/3526503751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdee0100040 0x7fdee0195430 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 >> 172.21.15.42:0/3526503751 conn(0x7fdee00fb8d0 msgr2=0x7fdee0101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 shutdown_connections 2023-12-12T09:51:00.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:00.281+0000 7fdee5332700 1 -- 172.21.15.42:0/3526503751 wait complete. 2023-12-12T09:51:00.687 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch host ls --format=json 2023-12-12T09:51:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:00 smithi042 ceph-mon[98894]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:51:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:00 smithi042 ceph-mon[98894]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:00 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:00 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:00 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:00 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:00 smithi042 ceph-mon[98894]: Added host smithi173 2023-12-12T09:51:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:00 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:00 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:01.296 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:51:02.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.679+0000 7f1f49dc4700 1 -- 172.21.15.42:0/2040851774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 msgr2=0x7f1f44100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:02.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.679+0000 7f1f49dc4700 1 --2- 172.21.15.42:0/2040851774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 0x7f1f44100410 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009910 tx=0x7f1f34009c20 comp rx=0 tx=0).stop 2023-12-12T09:51:02.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.679+0000 7f1f49dc4700 1 -- 172.21.15.42:0/2040851774 shutdown_connections 2023-12-12T09:51:02.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.679+0000 7f1f49dc4700 1 --2- 172.21.15.42:0/2040851774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 0x7f1f44100410 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:02.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.679+0000 7f1f49dc4700 1 -- 172.21.15.42:0/2040851774 >> 172.21.15.42:0/2040851774 conn(0x7f1f440fb900 msgr2=0x7f1f440fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:02.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.679+0000 7f1f49dc4700 1 -- 172.21.15.42:0/2040851774 shutdown_connections 2023-12-12T09:51:02.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.679+0000 7f1f49dc4700 1 -- 172.21.15.42:0/2040851774 wait complete. 2023-12-12T09:51:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.680+0000 7f1f49dc4700 1 Processor -- start 2023-12-12T09:51:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.680+0000 7f1f49dc4700 1 -- start start 2023-12-12T09:51:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.680+0000 7f1f49dc4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 0x7f1f44197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.680+0000 7f1f49dc4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4406a2e0 con 0x7f1f44100030 2023-12-12T09:51:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.680+0000 7f1f437fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 0x7f1f44197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.680+0000 7f1f437fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 0x7f1f44197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60716/0 (socket says 172.21.15.42:60716) 2023-12-12T09:51:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.681+0000 7f1f437fe700 1 -- 172.21.15.42:0/1957421260 learned_addr learned my addr 172.21.15.42:0/1957421260 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:51:02.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.681+0000 7f1f437fe700 1 -- 172.21.15.42:0/1957421260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f340095f0 con 0x7f1f44100030 2023-12-12T09:51:02.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.681+0000 7f1f437fe700 1 --2- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 0x7f1f44197670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1f34006010 tx=0x7f1f34004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:02.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.681+0000 7f1f40ff9700 1 -- 172.21.15.42:0/1957421260 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f3401d070 con 0x7f1f44100030 2023-12-12T09:51:02.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.681+0000 7f1f40ff9700 1 -- 172.21.15.42:0/1957421260 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f3400bc10 con 0x7f1f44100030 2023-12-12T09:51:02.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.681+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f44197c10 con 0x7f1f44100030 2023-12-12T09:51:02.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.681+0000 7f1f40ff9700 1 -- 172.21.15.42:0/1957421260 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f340038a0 con 0x7f1f44100030 2023-12-12T09:51:02.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.681+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f44198090 con 0x7f1f44100030 2023-12-12T09:51:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.682+0000 7f1f40ff9700 1 -- 172.21.15.42:0/1957421260 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f1f34003ad0 con 0x7f1f44100030 2023-12-12T09:51:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.682+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f44107fd0 con 0x7f1f44100030 2023-12-12T09:51:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.682+0000 7f1f40ff9700 1 --2- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1f2c0388c0 0x7f1f2c03ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.682+0000 7f1f40ff9700 1 -- 172.21.15.42:0/1957421260 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1f3404c670 con 0x7f1f44100030 2023-12-12T09:51:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.683+0000 7f1f42ffd700 1 --2- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1f2c0388c0 0x7f1f2c03ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:02.685 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.683+0000 7f1f42ffd700 1 --2- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1f2c0388c0 0x7f1f2c03ad80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006fd0 tx=0x7f1f38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:02.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.687+0000 7f1f40ff9700 1 -- 172.21.15.42:0/1957421260 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1f3402ad80 con 0x7f1f44100030 2023-12-12T09:51:02.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.896+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f44198280 con 0x7f1f2c0388c0 2023-12-12T09:51:02.899 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.897+0000 7f1f40ff9700 1 -- 172.21.15.42:0/1957421260 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7f1f44198280 con 0x7f1f2c0388c0 2023-12-12T09:51:02.899 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:51:02.899 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.111", "hostname": "smithi111", "labels": [], "status": ""}, {"addr": "172.21.15.173", "hostname": "smithi173", "labels": [], "status": ""}] 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1f2c0388c0 msgr2=0x7f1f2c03ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 --2- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1f2c0388c0 0x7f1f2c03ad80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1f38006fd0 tx=0x7f1f38006e40 comp rx=0 tx=0).stop 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 msgr2=0x7f1f44197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 --2- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 0x7f1f44197670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1f34006010 tx=0x7f1f34004ff0 comp rx=0 tx=0).stop 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 shutdown_connections 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 --2- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1f2c0388c0 0x7f1f2c03ad80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 --2- 172.21.15.42:0/1957421260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f44100030 0x7f1f44197670 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 >> 172.21.15.42:0/1957421260 conn(0x7f1f440fb900 msgr2=0x7f1f44101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 shutdown_connections 2023-12-12T09:51:02.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:02.899+0000 7f1f49dc4700 1 -- 172.21.15.42:0/1957421260 wait complete. 2023-12-12T09:51:02.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:02 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:03.266 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-12T09:51:03.266 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd crush tunables default 2023-12-12T09:51:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:03 smithi042 ceph-mon[98894]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:03 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:03.869 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:51:04.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:04 smithi042 ceph-mon[98894]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:51:05.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.206+0000 7f7b68f5c700 1 -- 172.21.15.42:0/3570606441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 msgr2=0x7f7b64100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:05.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.206+0000 7f7b68f5c700 1 --2- 172.21.15.42:0/3570606441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 0x7f7b64100430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7b54009910 tx=0x7f7b54009c20 comp rx=0 tx=0).stop 2023-12-12T09:51:05.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.206+0000 7f7b68f5c700 1 -- 172.21.15.42:0/3570606441 shutdown_connections 2023-12-12T09:51:05.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.206+0000 7f7b68f5c700 1 --2- 172.21.15.42:0/3570606441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 0x7f7b64100430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:05.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.206+0000 7f7b68f5c700 1 -- 172.21.15.42:0/3570606441 >> 172.21.15.42:0/3570606441 conn(0x7f7b640fb900 msgr2=0x7f7b640fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:05.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.206+0000 7f7b68f5c700 1 -- 172.21.15.42:0/3570606441 shutdown_connections 2023-12-12T09:51:05.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.206+0000 7f7b68f5c700 1 -- 172.21.15.42:0/3570606441 wait complete. 2023-12-12T09:51:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.206+0000 7f7b68f5c700 1 Processor -- start 2023-12-12T09:51:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b68f5c700 1 -- start start 2023-12-12T09:51:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b68f5c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 0x7f7b64198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:05.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b68f5c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64198f50 con 0x7f7b64100050 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b6259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 0x7f7b64198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b6259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 0x7f7b64198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60722/0 (socket says 172.21.15.42:60722) 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b6259c700 1 -- 172.21.15.42:0/35755472 learned_addr learned my addr 172.21.15.42:0/35755472 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b6259c700 1 -- 172.21.15.42:0/35755472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540095f0 con 0x7f7b64100050 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b6259c700 1 --2- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 0x7f7b64198a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7b54006010 tx=0x7f7b54004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b5b7fe700 1 -- 172.21.15.42:0/35755472 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b5401d070 con 0x7f7b64100050 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b64199150 con 0x7f7b64100050 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b5b7fe700 1 -- 172.21.15.42:0/35755472 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b5400bc10 con 0x7f7b64100050 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.207+0000 7f7b5b7fe700 1 -- 172.21.15.42:0/35755472 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b540038a0 con 0x7f7b64100050 2023-12-12T09:51:05.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.208+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b64199570 con 0x7f7b64100050 2023-12-12T09:51:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.208+0000 7f7b5b7fe700 1 -- 172.21.15.42:0/35755472 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f7b5400b6f0 con 0x7f7b64100050 2023-12-12T09:51:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.209+0000 7f7b5b7fe700 1 --2- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b500388c0 0x7f7b5003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.209+0000 7f7b5b7fe700 1 -- 172.21.15.42:0/35755472 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7b5404c560 con 0x7f7b64100050 2023-12-12T09:51:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.209+0000 7f7b61d9b700 1 --2- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b500388c0 0x7f7b5003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:05.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.209+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b64107fc0 con 0x7f7b64100050 2023-12-12T09:51:05.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.209+0000 7f7b61d9b700 1 --2- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b500388c0 0x7f7b5003ad80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c006fd0 tx=0x7f7b4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:05.215 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.214+0000 7f7b5b7fe700 1 -- 172.21.15.42:0/35755472 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7b54026020 con 0x7f7b64100050 2023-12-12T09:51:05.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.419+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7b64199d20 con 0x7f7b64100050 2023-12-12T09:51:05.471 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.470+0000 7f7b5b7fe700 1 -- 172.21.15.42:0/35755472 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7b5400bd80 con 0x7f7b64100050 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b500388c0 msgr2=0x7f7b5003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 --2- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b500388c0 0x7f7b5003ad80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c006fd0 tx=0x7f7b4c006e40 comp rx=0 tx=0).stop 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 msgr2=0x7f7b64198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 --2- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 0x7f7b64198a10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7b54006010 tx=0x7f7b54004ff0 comp rx=0 tx=0).stop 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 shutdown_connections 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 --2- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b500388c0 0x7f7b5003ad80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 --2- 172.21.15.42:0/35755472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b64100050 0x7f7b64198a10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 >> 172.21.15.42:0/35755472 conn(0x7f7b640fb900 msgr2=0x7f7b64068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 shutdown_connections 2023-12-12T09:51:05.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:05.472+0000 7f7b68f5c700 1 -- 172.21.15.42:0/35755472 wait complete. 2023-12-12T09:51:05.475 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2023-12-12T09:51:05.829 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:05 smithi042 ceph-mon[98894]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:05.831 INFO:tasks.cephadm:Adding mon.a on smithi042 2023-12-12T09:51:05.831 INFO:tasks.cephadm:Adding mon.b on smithi111 2023-12-12T09:51:05.831 INFO:tasks.cephadm:Adding mon.c on smithi173 2023-12-12T09:51:05.831 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch apply mon '3;smithi042:172.21.15.42=a;smithi111:172.21.15.111=b;smithi173:172.21.15.173=c' 2023-12-12T09:51:06.462 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/35755472' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/35755472' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:07.673 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.671+0000 7fb90d4f6700 1 -- 172.21.15.173:0/352460929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 msgr2=0x7fb908101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:07.673 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.671+0000 7fb90d4f6700 1 --2- 172.21.15.173:0/352460929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 0x7fb908101540 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009910 tx=0x7fb8f8009c20 comp rx=0 tx=0).stop 2023-12-12T09:51:07.673 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.671+0000 7fb90d4f6700 1 -- 172.21.15.173:0/352460929 shutdown_connections 2023-12-12T09:51:07.673 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.671+0000 7fb90d4f6700 1 --2- 172.21.15.173:0/352460929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 0x7fb908101540 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:07.673 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.671+0000 7fb90d4f6700 1 -- 172.21.15.173:0/352460929 >> 172.21.15.173:0/352460929 conn(0x7fb9080fca30 msgr2=0x7fb9080fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:07.673 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.671+0000 7fb90d4f6700 1 -- 172.21.15.173:0/352460929 shutdown_connections 2023-12-12T09:51:07.673 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.671+0000 7fb90d4f6700 1 -- 172.21.15.173:0/352460929 wait complete. 2023-12-12T09:51:07.674 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.672+0000 7fb90d4f6700 1 Processor -- start 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.672+0000 7fb90d4f6700 1 -- start start 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.672+0000 7fb90d4f6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 0x7fb90819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.672+0000 7fb90d4f6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90819ca90 con 0x7fb908101160 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.672+0000 7fb906ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 0x7fb90819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.672+0000 7fb906ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 0x7fb90819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.173:48700/0 (socket says 172.21.15.173:48700) 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.672+0000 7fb906ffd700 1 -- 172.21.15.173:0/562968227 learned_addr learned my addr 172.21.15.173:0/562968227 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.673+0000 7fb906ffd700 1 -- 172.21.15.173:0/562968227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80095f0 con 0x7fb908101160 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.673+0000 7fb906ffd700 1 --2- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 0x7fb90819c550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8004f40 tx=0x7fb8f8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.673+0000 7fb8effff700 1 -- 172.21.15.173:0/562968227 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f801d070 con 0x7fb908101160 2023-12-12T09:51:07.675 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.673+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb90819cc90 con 0x7fb908101160 2023-12-12T09:51:07.676 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.673+0000 7fb8effff700 1 -- 172.21.15.173:0/562968227 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8f800bc10 con 0x7fb908101160 2023-12-12T09:51:07.676 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.673+0000 7fb8effff700 1 -- 172.21.15.173:0/562968227 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8f80037d0 con 0x7fb908101160 2023-12-12T09:51:07.676 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.673+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90819d0b0 con 0x7fb908101160 2023-12-12T09:51:07.676 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.674+0000 7fb8effff700 1 -- 172.21.15.173:0/562968227 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fb8f800b6f0 con 0x7fb908101160 2023-12-12T09:51:07.677 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.674+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb908102640 con 0x7fb908101160 2023-12-12T09:51:07.677 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.674+0000 7fb8effff700 1 --2- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb8f00388c0 0x7fb8f003ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:07.677 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.674+0000 7fb8effff700 1 -- 172.21.15.173:0/562968227 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb8f804c5d0 con 0x7fb908101160 2023-12-12T09:51:07.677 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.675+0000 7fb9067fc700 1 --2- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb8f00388c0 0x7fb8f003ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:07.677 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.675+0000 7fb9067fc700 1 --2- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb8f00388c0 0x7fb8f003ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006fd0 tx=0x7fb8fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:07.681 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.680+0000 7fb8effff700 1 -- 172.21.15.173:0/562968227 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb8f802ad40 con 0x7fb908101160 2023-12-12T09:51:07.880 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.879+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi111:172.21.15.111=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fb90810b310 con 0x7fb8f00388c0 2023-12-12T09:51:07.888 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.886+0000 7fb8effff700 1 -- 172.21.15.173:0/562968227 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fb90810b310 con 0x7fb8f00388c0 2023-12-12T09:51:07.888 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mon update... 2023-12-12T09:51:07.890 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb8f00388c0 msgr2=0x7fb8f003ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:07.890 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 --2- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb8f00388c0 0x7fb8f003ad80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006fd0 tx=0x7fb8fc006e40 comp rx=0 tx=0).stop 2023-12-12T09:51:07.890 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 msgr2=0x7fb90819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:07.890 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 --2- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 0x7fb90819c550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8004f40 tx=0x7fb8f8005020 comp rx=0 tx=0).stop 2023-12-12T09:51:07.890 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 shutdown_connections 2023-12-12T09:51:07.890 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 --2- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb8f00388c0 0x7fb8f003ad80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:07.890 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 --2- 172.21.15.173:0/562968227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb908101160 0x7fb90819c550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:07.890 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 >> 172.21.15.173:0/562968227 conn(0x7fb9080fca30 msgr2=0x7fb908102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:07.891 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 shutdown_connections 2023-12-12T09:51:07.891 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:07.888+0000 7fb90d4f6700 1 -- 172.21.15.173:0/562968227 wait complete. 2023-12-12T09:51:07.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:07 smithi042 ceph-mon[98894]: Updating smithi173:/etc/ceph/ceph.conf 2023-12-12T09:51:07.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:07 smithi042 ceph-mon[98894]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:07 smithi042 ceph-mon[98894]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:07 smithi042 ceph-mon[98894]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:51:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:07 smithi042 ceph-mon[98894]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:51:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:07 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:07 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:07 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:08.325 DEBUG:teuthology.orchestra.run.smithi111:mon.b> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.b.service 2023-12-12T09:51:08.329 DEBUG:teuthology.orchestra.run.smithi173:mon.c> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.c.service 2023-12-12T09:51:08.333 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-12T09:51:08.333 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph mon dump -f json 2023-12-12T09:51:08.362 INFO:journalctl@ceph.mon.c.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:51:08.365 INFO:journalctl@ceph.mon.b.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi111:172.21.15.111=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi111:172.21.15.111=b;smithi173:172.21.15.173=c;count:3 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:08 smithi042 ceph-mon[98894]: Deploying daemon mon.c on smithi173 2023-12-12T09:51:09.227 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:10.900 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.896+0000 7f08455e0700 1 -- 172.21.15.173:0/79797867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 msgr2=0x7f0840106250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:10.900 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.896+0000 7f08455e0700 1 --2- 172.21.15.173:0/79797867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 0x7f0840106250 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0830009910 tx=0x7f0830009c20 comp rx=0 tx=0).stop 2023-12-12T09:51:10.900 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.896+0000 7f08455e0700 1 -- 172.21.15.173:0/79797867 shutdown_connections 2023-12-12T09:51:10.900 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.896+0000 7f08455e0700 1 --2- 172.21.15.173:0/79797867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 0x7f0840106250 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:10.900 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.896+0000 7f08455e0700 1 -- 172.21.15.173:0/79797867 >> 172.21.15.173:0/79797867 conn(0x7f0840075930 msgr2=0x7f0840075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:10.900 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.896+0000 7f08455e0700 1 -- 172.21.15.173:0/79797867 shutdown_connections 2023-12-12T09:51:10.900 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.896+0000 7f08455e0700 1 -- 172.21.15.173:0/79797867 wait complete. 2023-12-12T09:51:10.900 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.897+0000 7f08455e0700 1 Processor -- start 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.897+0000 7f08455e0700 1 -- start start 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.897+0000 7f08455e0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 0x7f08401976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.897+0000 7f08455e0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08400696b0 con 0x7f0840105e70 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.897+0000 7f083effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 0x7f08401976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.897+0000 7f083effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 0x7f08401976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.173:48706/0 (socket says 172.21.15.173:48706) 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.897+0000 7f083effd700 1 -- 172.21.15.173:0/391147455 learned_addr learned my addr 172.21.15.173:0/391147455 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.898+0000 7f083effd700 1 -- 172.21.15.173:0/391147455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08300095f0 con 0x7f0840105e70 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.898+0000 7f083effd700 1 --2- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 0x7f08401976a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0830006010 tx=0x7f0830004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.898+0000 7f0827fff700 1 -- 172.21.15.173:0/391147455 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f083001d070 con 0x7f0840105e70 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.898+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0840197ca0 con 0x7f0840105e70 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.898+0000 7f0827fff700 1 -- 172.21.15.173:0/391147455 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f083000bc10 con 0x7f0840105e70 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.898+0000 7f0827fff700 1 -- 172.21.15.173:0/391147455 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0830003860 con 0x7f0840105e70 2023-12-12T09:51:10.901 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.898+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0840198120 con 0x7f0840105e70 2023-12-12T09:51:10.902 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.899+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0840101520 con 0x7f0840105e70 2023-12-12T09:51:10.902 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.899+0000 7f0827fff700 1 -- 172.21.15.173:0/391147455 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f083000b6f0 con 0x7f0840105e70 2023-12-12T09:51:10.902 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.899+0000 7f0827fff700 1 --2- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f08280388c0 0x7f082803ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:10.902 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.899+0000 7f0827fff700 1 -- 172.21.15.173:0/391147455 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f083004c540 con 0x7f0840105e70 2023-12-12T09:51:10.902 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.899+0000 7f083e7fc700 1 --2- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f08280388c0 0x7f082803ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:10.902 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.900+0000 7f083e7fc700 1 --2- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f08280388c0 0x7f082803ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0834006fd0 tx=0x7f0834006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:10.906 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:10.905+0000 7f0827fff700 1 -- 172.21.15.173:0/391147455 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0830004390 con 0x7f0840105e70 2023-12-12T09:51:11.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:10 smithi042 ceph-mon[98894]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:11.222 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.219+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0840198370 con 0x7f0840105e70 2023-12-12T09:51:11.222 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.220+0000 7f0827fff700 1 -- 172.21.15.173:0/391147455 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f0830026020 con 0x7f0840105e70 2023-12-12T09:51:11.223 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:51:11.223 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":1,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","modified":"2023-12-12T09:49:21.521913Z","created":"2023-12-12T09:49:21.521913Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-12T09:51:11.223 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f08280388c0 msgr2=0x7f082803ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 --2- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f08280388c0 0x7f082803ad80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0834006fd0 tx=0x7f0834006e40 comp rx=0 tx=0).stop 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 msgr2=0x7f08401976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 --2- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 0x7f08401976a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0830006010 tx=0x7f0830004ff0 comp rx=0 tx=0).stop 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 shutdown_connections 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 --2- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f08280388c0 0x7f082803ad80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 --2- 172.21.15.173:0/391147455 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0840105e70 0x7f08401976a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 >> 172.21.15.173:0/391147455 conn(0x7f0840075930 msgr2=0x7f08400fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 shutdown_connections 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:11.222+0000 7f08455e0700 1 -- 172.21.15.173:0/391147455 wait complete. 2023-12-12T09:51:11.224 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 1 2023-12-12T09:51:11.806 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 podman[103266]: 2023-12-12T09:51:12.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: set uid:gid to 167:167 (ceph:ceph) 2023-12-12T09:51:12.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev), process ceph-mon, pid 7 2023-12-12T09:51:12.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pidfile_write: ignore empty --pid-file 2023-12-12T09:51:12.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 bash[103266]: 347f73327bef2f17f70cad29d6ba6bc6924d9012ddd50ebbf5d63788ab556dbf 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: load: jerasure load: lrc 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: RocksDB version: 7.9.2 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Git sha 0 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Compile date 2023-12-11 04:34:03 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: DB SUMMARY 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: DB Session ID: AWSZ079TVMRVC05L911A 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: CURRENT file: CURRENT 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: IDENTITY file: IDENTITY 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.error_if_exists: 0 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.create_if_missing: 0 2023-12-12T09:51:12.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.paranoid_checks: 1 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.env: 0x5650325fca60 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.fs: PosixFileSystem 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.info_log: 0x565034e57740 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_file_opening_threads: 16 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.statistics: (nil) 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.use_fsync: 0 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_log_file_size: 0 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-12T09:51:12.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.keep_log_file_num: 1000 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.recycle_log_file_num: 0 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.allow_fallocate: 1 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.allow_mmap_reads: 0 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.allow_mmap_writes: 0 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.use_direct_reads: 0 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.create_missing_column_families: 0 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.db_log_dir: 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.wal_dir: 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-12T09:51:12.076 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.advise_random_on_open: 1 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.db_write_buffer_size: 0 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.write_buffer_manager: 0x565034e0d360 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.rate_limiter: (nil) 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.wal_recovery_mode: 2 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.enable_thread_tracking: 0 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.enable_pipelined_write: 0 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.unordered_write: 0 2023-12-12T09:51:12.077 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.row_cache: None 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.wal_filter: None 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.allow_ingest_behind: 0 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.two_write_queues: 0 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.manual_wal_flush: 0 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.wal_compression: 0 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.atomic_flush: 0 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-12T09:51:12.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.log_readahead_size: 0 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.best_efforts_recovery: 0 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.allow_data_in_errors: 0 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.db_host_id: __hostname__ 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_background_jobs: 2 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_background_compactions: -1 2023-12-12T09:51:12.079 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_subcompactions: 1 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_total_wal_size: 0 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_open_files: -1 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bytes_per_sync: 0 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_readahead_size: 0 2023-12-12T09:51:12.080 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_background_flushes: -1 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Compression algorithms supported: 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: kZSTD supported: 0 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: kXpressCompression supported: 0 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: kLZ4HCCompression supported: 1 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: kZlibCompression supported: 1 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: kSnappyCompression supported: 1 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: kLZ4Compression supported: 1 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: kBZip2Compression supported: 0 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-12T09:51:12.081 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.merge_operator: 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_filter: None 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_filter_factory: None 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.sst_partitioner_factory: None 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x565034e57860) 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout: cache_index_and_filter_blocks: 1 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout: pin_top_level_index_and_filter: 1 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout: index_type: 0 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout: data_block_index_type: 0 2023-12-12T09:51:12.082 INFO:journalctl@ceph.mon.c.smithi173.stdout: index_shortening: 1 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: checksum: 4 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: no_block_cache: 0 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_cache: 0x565034ead090 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_cache_name: BinnedLRUCache 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_cache_options: 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: capacity : 536870912 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: num_shard_bits : 4 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: strict_capacity_limit : 0 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: high_pri_pool_ratio: 0.000 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_cache_compressed: (nil) 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: persistent_cache: (nil) 2023-12-12T09:51:12.083 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_size: 4096 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_size_deviation: 10 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_restart_interval: 16 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: index_block_restart_interval: 1 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: metadata_block_size: 4096 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: partition_filters: 0 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: use_delta_encoding: 1 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: filter_policy: bloomfilter 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: whole_key_filtering: 1 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: verify_compression: 0 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: read_amp_bytes_per_bit: 0 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: format_version: 5 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: enable_index_compression: 1 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: block_align: 0 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: max_auto_readahead_size: 262144 2023-12-12T09:51:12.084 INFO:journalctl@ceph.mon.c.smithi173.stdout: prepopulate_block_cache: 0 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout: initial_auto_readahead_size: 8192 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout: num_file_reads_for_auto_readahead: 2 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.write_buffer_size: 33554432 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_write_buffer_number: 2 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression: NoCompression 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression: Disabled 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.prefix_extractor: nullptr 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.num_levels: 7 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-12T09:51:12.085 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression_opts.level: 32767 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression_opts.strategy: 0 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-12T09:51:12.086 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression_opts.enabled: false 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.target_file_size_base: 67108864 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-12T09:51:12.087 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-12T09:51:12.088 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-12T09:51:12.088 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-12T09:51:12.088 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-12T09:51:12.088 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-12T09:51:12.088 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.arena_block_size: 1048576 2023-12-12T09:51:12.088 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-12T09:51:12.088 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-12T09:51:12.088 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.disable_auto_compactions: 0 2023-12-12T09:51:12.088 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-12T09:51:12.089 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-12T09:51:12.089 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-12T09:51:12.089 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-12T09:51:12.089 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-12T09:51:12.089 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.table_properties_collectors: 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.inplace_update_support: 0 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.bloom_locality: 0 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.max_successive_merges: 0 2023-12-12T09:51:12.090 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.paranoid_file_checks: 0 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.force_consistency_checks: 1 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.report_bg_io_stats: 0 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.ttl: 2592000 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.enable_blob_files: false 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.min_blob_size: 0 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.blob_file_size: 268435456 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-12T09:51:12.091 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.blob_file_starting_level: 0 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 27ae7646-5fc9-4fc6-8335-95192b6b6111 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702374671811101, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702374671811511, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702374671, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "27ae7646-5fc9-4fc6-8335-95192b6b6111", "db_session_id": "AWSZ079TVMRVC05L911A", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702374671811577, "job": 1, "event": "recovery_finished"} 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x565034f48000 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: DB pointer 0x565034f34000 2023-12-12T09:51:12.092 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: ** DB Stats ** 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: ** Compaction Stats [default] ** 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-12T09:51:12.093 INFO:journalctl@ceph.mon.c.smithi173.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: ** Compaction Stats [default] ** 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-12T09:51:12.094 INFO:journalctl@ceph.mon.c.smithi173.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout: Cumulative compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout: Interval compaction: 0.00 GB write, 0.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout: Block cache BinnedLRUCache@0x565034ead090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: using public_addr v2:172.21.15.173:0/0 -> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: starting mon.c rank -1 at public addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] at bind addrs [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(???) e0 preinit fsid 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).mds e1 new map 2023-12-12T09:51:12.095 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).mds e1 print_map 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout: e1 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout: legacy client fscid: -1 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout: 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout: No filesystems configured 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-12T09:51:12.096 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Updating smithi111:/etc/ceph/ceph.conf 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Deploying cephadm binary to smithi173 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:12.097 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Added host smithi173 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:12.098 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/35755472' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/35755472' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Updating smithi173:/etc/ceph/ceph.conf 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:12.099 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi111:172.21.15.111=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi111:172.21.15.111=b;smithi173:172.21.15.173=c;count:3 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:12.100 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:12.101 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: Deploying daemon mon.c on smithi173 2023-12-12T09:51:12.101 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:12.101 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 ceph-mon[103308]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-12T09:51:12.101 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:11 smithi173 systemd[1]: Started Ceph mon.c for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:51:12.695 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-12T09:51:12.695 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph mon dump -f json 2023-12-12T09:51:13.295 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:51:17.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:17.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:51:17.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: mon.a calling monitor election 2023-12-12T09:51:17.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: mon.c calling monitor election 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: monmap e2: 2 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],c=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: fsmap 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: mgrmap e13: x(active, since 49s) 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: overall HEALTH_OK 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:17.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:17.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:17.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:17.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:51:17.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: mon.a calling monitor election 2023-12-12T09:51:17.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: mon.c calling monitor election 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: monmap e2: 2 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],c=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: fsmap 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: mgrmap e13: x(active, since 49s) 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: overall HEALTH_OK 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:17.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:17.496 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.493+0000 7f02431af700 1 -- 172.21.15.173:0/3537989307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f022c0035c0 msgr2=0x7f022c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:17.496 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.493+0000 7f02431af700 1 --2- 172.21.15.173:0/3537989307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f022c0035c0 0x7f022c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f023c0ff5a0 tx=0x7f0230010ec0 comp rx=0 tx=0).stop 2023-12-12T09:51:17.496 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.493+0000 7f02431af700 1 -- 172.21.15.173:0/3537989307 shutdown_connections 2023-12-12T09:51:17.496 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.493+0000 7f02431af700 1 --2- 172.21.15.173:0/3537989307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f022c0035c0 0x7f022c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:17.496 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.493+0000 7f02431af700 1 -- 172.21.15.173:0/3537989307 >> 172.21.15.173:0/3537989307 conn(0x7f023c0765f0 msgr2=0x7f023c076a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:17.496 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.493+0000 7f02431af700 1 -- 172.21.15.173:0/3537989307 shutdown_connections 2023-12-12T09:51:17.496 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.493+0000 7f02431af700 1 -- 172.21.15.173:0/3537989307 wait complete. 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.494+0000 7f02431af700 1 Processor -- start 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f02431af700 1 -- start start 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f02431af700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f022c0035c0 0x7f023c19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f02431af700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 0x7f023c1966e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f02431af700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c19cf60 con 0x7f022c0035c0 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f02431af700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c19d0d0 con 0x7f023c19cae0 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f023bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 0x7f023c1966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f023bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 0x7f023c1966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42442/0 (socket says 172.21.15.173:42442) 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f023bfff700 1 -- 172.21.15.173:0/2909441755 learned_addr learned my addr 172.21.15.173:0/2909441755 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f0240f4b700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f022c0035c0 0x7f023c19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:17.498 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f023bfff700 1 -- 172.21.15.173:0/2909441755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 msgr2=0x7f023c1966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-12-12T09:51:17.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f023bfff700 1 -- 172.21.15.173:0/2909441755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 msgr2=0x7f023c1966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-12T09:51:17.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f023bfff700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 0x7f023c1966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-12T09:51:17.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.495+0000 7f023bfff700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 0x7f023c1966e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:51:17.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.496+0000 7f0240f4b700 1 -- 172.21.15.173:0/2909441755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 msgr2=0x7f023c1966e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:51:17.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.496+0000 7f0240f4b700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 0x7f023c1966e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:17.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.496+0000 7f0240f4b700 1 -- 172.21.15.173:0/2909441755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230010910 con 0x7f022c0035c0 2023-12-12T09:51:17.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.496+0000 7f0240f4b700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f022c0035c0 0x7f023c19c5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0230000c00 tx=0x7f0230010790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:17.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.496+0000 7f0239ffb700 1 -- 172.21.15.173:0/2909441755 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0230017050 con 0x7f022c0035c0 2023-12-12T09:51:17.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.496+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023c196ce0 con 0x7f022c0035c0 2023-12-12T09:51:17.500 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.496+0000 7f0239ffb700 1 -- 172.21.15.173:0/2909441755 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02300044f0 con 0x7f022c0035c0 2023-12-12T09:51:17.500 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.496+0000 7f0239ffb700 1 -- 172.21.15.173:0/2909441755 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f0230003d00 con 0x7f022c0035c0 2023-12-12T09:51:17.500 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.497+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023c1971e0 con 0x7f022c0035c0 2023-12-12T09:51:17.501 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.498+0000 7f0239ffb700 1 -- 172.21.15.173:0/2909441755 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f0230003f70 con 0x7f022c0035c0 2023-12-12T09:51:17.501 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.498+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f023c0fecf0 con 0x7f022c0035c0 2023-12-12T09:51:17.501 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.498+0000 7f0239ffb700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0220038700 0x7f022003abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:17.501 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.498+0000 7f0239ffb700 1 -- 172.21.15.173:0/2909441755 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f023004dd40 con 0x7f022c0035c0 2023-12-12T09:51:17.501 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.498+0000 7f023bfff700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0220038700 0x7f022003abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:17.501 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.499+0000 7f023bfff700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0220038700 0x7f022003abc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0228009180 tx=0x7f0228008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:17.505 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.504+0000 7f0239ffb700 1 -- 172.21.15.173:0/2909441755 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f023002d9b0 con 0x7f022c0035c0 2023-12-12T09:51:17.775 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.773+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f023c100db0 con 0x7f022c0035c0 2023-12-12T09:51:17.775 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.774+0000 7f0239ffb700 1 -- 172.21.15.173:0/2909441755 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f023002ecb0 con 0x7f022c0035c0 2023-12-12T09:51:17.776 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:51:17.776 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":2,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","modified":"2023-12-12T09:51:11.841453Z","created":"2023-12-12T09:49:21.521913Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-12T09:51:17.778 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0220038700 msgr2=0x7f022003abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:17.778 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0220038700 0x7f022003abc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0228009180 tx=0x7f0228008040 comp rx=0 tx=0).stop 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f022c0035c0 msgr2=0x7f023c19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f022c0035c0 0x7f023c19c5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0230000c00 tx=0x7f0230010790 comp rx=0 tx=0).stop 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 shutdown_connections 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0220038700 0x7f022003abc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f022c0035c0 0x7f023c19c5a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 --2- 172.21.15.173:0/2909441755 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f023c19cae0 0x7f023c1966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 >> 172.21.15.173:0/2909441755 conn(0x7f023c0765f0 msgr2=0x7f023c1025d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 shutdown_connections 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:17.776+0000 7f02431af700 1 -- 172.21.15.173:0/2909441755 wait complete. 2023-12-12T09:51:17.779 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 2 2023-12-12T09:51:18.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:17 smithi173 ceph-mon[103308]: Deploying daemon mon.b on smithi111 2023-12-12T09:51:18.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:17 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/2909441755' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-12T09:51:18.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:18.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:17 smithi042 ceph-mon[98894]: Deploying daemon mon.b on smithi111 2023-12-12T09:51:18.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:17 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/2909441755' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-12T09:51:18.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:19.260 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-12T09:51:19.260 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph mon dump -f json 2023-12-12T09:51:19.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:19 smithi173 ceph-mon[103308]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:19.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:19 smithi042 ceph-mon[98894]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 systemd[1]: Starting Ceph mon.b for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:51:19.862 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:51:20.019 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 podman[102525]: 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 bash[102525]: dc6d9382b8ef7f84221755da7b469d4cc4326da145569b70373b44d6fa1dd089 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: set uid:gid to 167:167 (ceph:ceph) 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev), process ceph-mon, pid 7 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pidfile_write: ignore empty --pid-file 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: load: jerasure load: lrc 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: RocksDB version: 7.9.2 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Git sha 0 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Compile date 2023-12-11 04:34:03 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: DB SUMMARY 2023-12-12T09:51:20.020 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: DB Session ID: 98TIX5C18ENSQSTGC025 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: CURRENT file: CURRENT 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: IDENTITY file: IDENTITY 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.error_if_exists: 0 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.create_if_missing: 0 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.paranoid_checks: 1 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.env: 0x55f04c85ea60 2023-12-12T09:51:20.021 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.fs: PosixFileSystem 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.info_log: 0x55f04dafd740 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_file_opening_threads: 16 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.statistics: (nil) 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.use_fsync: 0 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_log_file_size: 0 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.keep_log_file_num: 1000 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.recycle_log_file_num: 0 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.allow_fallocate: 1 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.allow_mmap_reads: 0 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.allow_mmap_writes: 0 2023-12-12T09:51:20.022 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.use_direct_reads: 0 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.create_missing_column_families: 0 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.db_log_dir: 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.wal_dir: 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.advise_random_on_open: 1 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.db_write_buffer_size: 0 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.write_buffer_manager: 0x55f04dab3360 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-12T09:51:20.023 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.rate_limiter: (nil) 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.wal_recovery_mode: 2 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.enable_thread_tracking: 0 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.enable_pipelined_write: 0 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.unordered_write: 0 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.row_cache: None 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.wal_filter: None 2023-12-12T09:51:20.024 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.allow_ingest_behind: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.two_write_queues: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.manual_wal_flush: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.wal_compression: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.atomic_flush: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.log_readahead_size: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.best_efforts_recovery: 0 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-12T09:51:20.025 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.allow_data_in_errors: 0 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.db_host_id: __hostname__ 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_background_jobs: 2 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_background_compactions: -1 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_subcompactions: 1 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_total_wal_size: 0 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-12T09:51:20.026 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_open_files: -1 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bytes_per_sync: 0 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_readahead_size: 0 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_background_flushes: -1 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Compression algorithms supported: 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: kZSTD supported: 0 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: kXpressCompression supported: 0 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: kLZ4HCCompression supported: 1 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: kZlibCompression supported: 1 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: kSnappyCompression supported: 1 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: kLZ4Compression supported: 1 2023-12-12T09:51:20.027 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: kBZip2Compression supported: 0 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.merge_operator: 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_filter: None 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_filter_factory: None 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.sst_partitioner_factory: None 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f04dafd860) 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout: cache_index_and_filter_blocks: 1 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-12T09:51:20.028 INFO:journalctl@ceph.mon.b.smithi111.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: pin_top_level_index_and_filter: 1 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: index_type: 0 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: data_block_index_type: 0 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: index_shortening: 1 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: checksum: 4 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: no_block_cache: 0 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_cache: 0x55f04db53090 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_cache_name: BinnedLRUCache 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_cache_options: 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: capacity : 536870912 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: num_shard_bits : 4 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: strict_capacity_limit : 0 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: high_pri_pool_ratio: 0.000 2023-12-12T09:51:20.029 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_cache_compressed: (nil) 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: persistent_cache: (nil) 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_size: 4096 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_size_deviation: 10 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_restart_interval: 16 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: index_block_restart_interval: 1 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: metadata_block_size: 4096 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: partition_filters: 0 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: use_delta_encoding: 1 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: filter_policy: bloomfilter 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: whole_key_filtering: 1 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: verify_compression: 0 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: read_amp_bytes_per_bit: 0 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: format_version: 5 2023-12-12T09:51:20.030 INFO:journalctl@ceph.mon.b.smithi111.stdout: enable_index_compression: 1 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout: block_align: 0 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout: max_auto_readahead_size: 262144 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout: prepopulate_block_cache: 0 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout: initial_auto_readahead_size: 8192 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout: num_file_reads_for_auto_readahead: 2 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.write_buffer_size: 33554432 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_write_buffer_number: 2 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression: NoCompression 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression: Disabled 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.prefix_extractor: nullptr 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.num_levels: 7 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-12T09:51:20.031 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-12T09:51:20.032 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-12T09:51:20.032 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-12T09:51:20.032 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-12T09:51:20.033 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-12T09:51:20.033 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-12T09:51:20.033 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-12T09:51:20.033 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-12T09:51:20.033 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-12T09:51:20.033 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-12T09:51:20.033 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-12T09:51:20.033 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression_opts.level: 32767 2023-12-12T09:51:20.033 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression_opts.strategy: 0 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression_opts.enabled: false 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.target_file_size_base: 67108864 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-12T09:51:20.034 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.arena_block_size: 1048576 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.disable_auto_compactions: 0 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-12T09:51:20.035 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.table_properties_collectors: 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.inplace_update_support: 0 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-12T09:51:20.036 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.bloom_locality: 0 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.max_successive_merges: 0 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.paranoid_file_checks: 0 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.force_consistency_checks: 1 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.report_bg_io_stats: 0 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.ttl: 2592000 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.enable_blob_files: false 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.min_blob_size: 0 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.blob_file_size: 268435456 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-12T09:51:20.037 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.blob_file_starting_level: 0 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 31f4d3dd-d732-4035-9cf1-3a75d4bb619b 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702374679934734, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702374679935142, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702374679, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "31f4d3dd-d732-4035-9cf1-3a75d4bb619b", "db_session_id": "98TIX5C18ENSQSTGC025", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702374679935210, "job": 1, "event": "recovery_finished"} 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-12T09:51:20.038 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f04dbee000 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: DB pointer 0x55f04dbda000 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: ** DB Stats ** 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2023-12-12T09:51:20.039 INFO:journalctl@ceph.mon.b.smithi111.stdout: ** Compaction Stats [default] ** 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: ** Compaction Stats [default] ** 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-12T09:51:20.040 INFO:journalctl@ceph.mon.b.smithi111.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: Cumulative compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: Interval compaction: 0.00 GB write, 0.71 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: Block cache BinnedLRUCache@0x55f04db53090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: using public_addr v2:172.21.15.111:0/0 -> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] 2023-12-12T09:51:20.041 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: starting mon.b rank -1 at public addrs [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] at bind addrs [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(???) e0 preinit fsid 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).mds e1 new map 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).mds e1 print_map 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout: e1 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout: legacy client fscid: -1 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout: 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout: No filesystems configured 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-12T09:51:20.042 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi111", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi173", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Updating smithi111:/etc/ceph/ceph.conf 2023-12-12T09:51:20.043 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Deploying cephadm binary to smithi173 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Added host smithi173 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.044 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/35755472' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/35755472' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi173", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Updating smithi173:/etc/ceph/ceph.conf 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.045 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi111:172.21.15.111=b;smithi173:172.21.15.173=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi111:172.21.15.111=b;smithi173:172.21.15.173=c;count:3 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:20.046 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Deploying daemon mon.c on smithi173 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.a calling monitor election 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.c calling monitor election 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:20.047 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: monmap e2: 2 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],c=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: fsmap 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mgrmap e13: x(active, since 49s) 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: overall HEALTH_OK 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: Deploying daemon mon.b on smithi111 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/2909441755' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:20.048 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:19 smithi111 ceph-mon[102565]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-12T09:51:20.049 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:20 smithi111 systemd[1]: Started Ceph mon.b for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:51:25.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: mon.c calling monitor election 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: mon.a calling monitor election 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-12T09:51:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0],c=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: fsmap 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: mgrmap e13: x(active, since 57s) 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: mon.b (rank 2) addr [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] is down (out of quorum) 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:25.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: mon.c calling monitor election 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: mon.a calling monitor election 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0],c=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: fsmap 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: mgrmap e13: x(active, since 57s) 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: mon.b (rank 2) addr [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] is down (out of quorum) 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:25.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:25.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:25.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:25.490 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.488+0000 7f7de29d4700 1 -- 172.21.15.173:0/3846668313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dc40035c0 msgr2=0x7f7dc4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:25.490 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.488+0000 7f7de29d4700 1 --2- 172.21.15.173:0/3846668313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dc40035c0 0x7f7dc4005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0ffe20 tx=0x7f7dcc00fe60 comp rx=0 tx=0).stop 2023-12-12T09:51:25.490 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.488+0000 7f7de29d4700 1 -- 172.21.15.173:0/3846668313 shutdown_connections 2023-12-12T09:51:25.490 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.488+0000 7f7de29d4700 1 --2- 172.21.15.173:0/3846668313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7dc40035c0 0x7f7dc4005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:25.490 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.488+0000 7f7de29d4700 1 -- 172.21.15.173:0/3846668313 >> 172.21.15.173:0/3846668313 conn(0x7f7ddc0fb9e0 msgr2=0x7f7ddc0fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:25.490 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.488+0000 7f7de29d4700 1 -- 172.21.15.173:0/3846668313 shutdown_connections 2023-12-12T09:51:25.490 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.488+0000 7f7de29d4700 1 -- 172.21.15.173:0/3846668313 wait complete. 2023-12-12T09:51:25.492 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7de29d4700 1 Processor -- start 2023-12-12T09:51:25.492 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7de29d4700 1 -- start start 2023-12-12T09:51:25.492 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7de29d4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dc40035c0 0x7f7ddc108f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:25.492 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7de29d4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 0x7f7ddc109930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:25.492 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7de29d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ddc1a2bc0 0x7f7ddc1a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:25.492 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7de29d4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1a55b0 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.492 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7de29d4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1a5720 con 0x7f7dc40035c0 2023-12-12T09:51:25.492 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7de29d4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1a58e0 con 0x7f7ddc1094b0 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7ddb7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 0x7f7ddc109930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7ddb7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 0x7f7ddc109930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42470/0 (socket says 172.21.15.173:42470) 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.489+0000 7f7ddb7fe700 1 -- 172.21.15.173:0/3768777871 learned_addr learned my addr 172.21.15.173:0/3768777871 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7ddb7fe700 1 -- 172.21.15.173:0/3768777871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 msgr2=0x7f7ddc109930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7ddb7fe700 1 -- 172.21.15.173:0/3768777871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 msgr2=0x7f7ddc109930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7ddb7fe700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 0x7f7ddc109930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7de0f71700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ddc1a2bc0 0x7f7ddc1a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7ddb7fe700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 0x7f7ddc109930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7ddbfff700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dc40035c0 0x7f7ddc108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7ddbfff700 1 -- 172.21.15.173:0/3768777871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dc40035c0 msgr2=0x7f7ddc108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7ddbfff700 1 -- 172.21.15.173:0/3768777871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dc40035c0 msgr2=0x7f7ddc108f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7ddbfff700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dc40035c0 0x7f7ddc108f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7de0f71700 1 -- 172.21.15.173:0/3768777871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 msgr2=0x7f7ddc109930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:51:25.493 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7ddbfff700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dc40035c0 0x7f7ddc108f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7de0f71700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 0x7f7ddc109930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7de0f71700 1 -- 172.21.15.173:0/3768777871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dc40035c0 msgr2=0x7f7ddc108f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7de0f71700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dc40035c0 0x7f7ddc108f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7de0f71700 1 -- 172.21.15.173:0/3768777871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dcc00f8b0 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7de0f71700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ddc1a2bc0 0x7f7ddc1a4fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400bd50 tx=0x7f7dd400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7dd97fa700 1 -- 172.21.15.173:0/3768777871 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400cd10 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7dd97fa700 1 -- 172.21.15.173:0/3768777871 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7dd400ce70 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7dd97fa700 1 -- 172.21.15.173:0/3768777871 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4012840 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc1a5bc0 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.494 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.490+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc1a60c0 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.495 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.491+0000 7f7dd97fa700 1 -- 172.21.15.173:0/3768777871 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f7dd40129a0 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.495 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.491+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ddc101630 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.495 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.492+0000 7f7dd97fa700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7dc8038d90 0x7f7dc803b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:25.495 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.492+0000 7f7dd97fa700 1 -- 172.21.15.173:0/3768777871 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7dd404aec0 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.495 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.492+0000 7f7ddbfff700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7dc8038d90 0x7f7dc803b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:25.495 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.492+0000 7f7ddbfff700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7dc8038d90 0x7f7dc803b250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc005d10 tx=0x7f7dcc005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:25.499 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.497+0000 7f7dd97fa700 1 -- 172.21.15.173:0/3768777871 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7dd4011d00 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.772 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.771+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7ddc107c40 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.773 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.772+0000 7f7dd97fa700 1 -- 172.21.15.173:0/3768777871 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1274 (secure 0 0 0) 0x7f7dd40112a0 con 0x7f7ddc1a2bc0 2023-12-12T09:51:25.773 INFO:teuthology.orchestra.run.smithi173.stdout: 2023-12-12T09:51:25.773 INFO:teuthology.orchestra.run.smithi173.stdout:{"epoch":3,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","modified":"2023-12-12T09:51:19.963181Z","created":"2023-12-12T09:49:21.521913Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:3300","nonce":0},{"type":"v1","addr":"172.21.15.173:6789","nonce":0}]},"addr":"172.21.15.173:6789/0","public_addr":"172.21.15.173:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-12T09:51:25.775 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7dc8038d90 msgr2=0x7f7dc803b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:25.775 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7dc8038d90 0x7f7dc803b250 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc005d10 tx=0x7f7dcc005c00 comp rx=0 tx=0).stop 2023-12-12T09:51:25.775 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ddc1a2bc0 msgr2=0x7f7ddc1a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ddc1a2bc0 0x7f7ddc1a4fb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400bd50 tx=0x7f7dd400be30 comp rx=0 tx=0).stop 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 shutdown_connections 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7dc8038d90 0x7f7dc803b250 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7dc40035c0 0x7f7ddc108f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7ddc1a2bc0 0x7f7ddc1a4fb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 --2- 172.21.15.173:0/3768777871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ddc1094b0 0x7f7ddc109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 >> 172.21.15.173:0/3768777871 conn(0x7f7ddc0fb9e0 msgr2=0x7f7ddc103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 shutdown_connections 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:25.774+0000 7f7de29d4700 1 -- 172.21.15.173:0/3768777871 wait complete. 2023-12-12T09:51:25.776 INFO:teuthology.orchestra.run.smithi173.stderr:dumped monmap epoch 3 2023-12-12T09:51:26.234 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-12-12T09:51:26.234 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph config generate-minimal-conf 2023-12-12T09:51:26.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:26 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:26.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:26 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/3768777871' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-12T09:51:26.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:26 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:26.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:26 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:26.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:26 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/3768777871' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-12T09:51:26.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:26 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:26.853 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:51:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:51:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: mon.c calling monitor election 2023-12-12T09:51:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: mon.a calling monitor election 2023-12-12T09:51:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:51:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-12T09:51:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0],c=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:51:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: fsmap 2023-12-12T09:51:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: mgrmap e13: x(active, since 57s) 2023-12-12T09:51:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2023-12-12T09:51:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2023-12-12T09:51:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2023-12-12T09:51:27.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: mon.b (rank 2) addr [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] is down (out of quorum) 2023-12-12T09:51:27.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:27.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:27.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:27.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:27.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:27.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/3768777871' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-12T09:51:27.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: mon.b calling monitor election 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Updating smithi042:/etc/ceph/ceph.conf 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Updating smithi111:/etc/ceph/ceph.conf 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Updating smithi173:/etc/ceph/ceph.conf 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: mon.b calling monitor election 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: mon.c calling monitor election 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: mon.a calling monitor election 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0],c=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:51:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: fsmap 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: mgrmap e13: x(active, since 59s) 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Cluster is now healthy 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: overall HEALTH_OK 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Reconfiguring mon.a (unknown last config time)... 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: Reconfiguring daemon mon.a on smithi042 2023-12-12T09:51:28.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.085 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: mon.b calling monitor election 2023-12-12T09:51:28.085 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Updating smithi042:/etc/ceph/ceph.conf 2023-12-12T09:51:28.085 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Updating smithi111:/etc/ceph/ceph.conf 2023-12-12T09:51:28.085 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Updating smithi173:/etc/ceph/ceph.conf 2023-12-12T09:51:28.085 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: mon.b calling monitor election 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: mon.c calling monitor election 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: mon.a calling monitor election 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0],c=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: fsmap 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: mgrmap e13: x(active, since 59s) 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Cluster is now healthy 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: overall HEALTH_OK 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:28.086 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Reconfiguring mon.a (unknown last config time)... 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:28.087 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: Reconfiguring daemon mon.a on smithi042 2023-12-12T09:51:28.088 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: mon.b calling monitor election 2023-12-12T09:51:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Updating smithi042:/etc/ceph/ceph.conf 2023-12-12T09:51:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Updating smithi111:/etc/ceph/ceph.conf 2023-12-12T09:51:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Updating smithi173:/etc/ceph/ceph.conf 2023-12-12T09:51:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: mon.b calling monitor election 2023-12-12T09:51:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: mon.c calling monitor election 2023-12-12T09:51:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: mon.a calling monitor election 2023-12-12T09:51:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: monmap e3: 3 mons at {a=[v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0],b=[v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0],c=[v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0]} removed_ranks: {} disallowed_leaders: {} 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: fsmap 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: osdmap e4: 0 total, 0 up, 0 in 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: mgrmap e13: x(active, since 59s) 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Cluster is now healthy 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: overall HEALTH_OK 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Reconfiguring mon.a (unknown last config time)... 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: Reconfiguring daemon mon.a on smithi042 2023-12-12T09:51:28.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:28.267 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.266+0000 7f96e1110700 1 -- 172.21.15.42:0/1475707713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 msgr2=0x7f96dc100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:28.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.266+0000 7f96e1110700 1 --2- 172.21.15.42:0/1475707713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 0x7f96dc100430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96d0009910 tx=0x7f96d0009c20 comp rx=0 tx=0).stop 2023-12-12T09:51:28.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.266+0000 7f96e1110700 1 -- 172.21.15.42:0/1475707713 shutdown_connections 2023-12-12T09:51:28.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.266+0000 7f96e1110700 1 --2- 172.21.15.42:0/1475707713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 0x7f96dc100430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:28.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.266+0000 7f96e1110700 1 -- 172.21.15.42:0/1475707713 >> 172.21.15.42:0/1475707713 conn(0x7f96dc0fb900 msgr2=0x7f96dc0fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:28.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.266+0000 7f96e1110700 1 -- 172.21.15.42:0/1475707713 shutdown_connections 2023-12-12T09:51:28.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.266+0000 7f96e1110700 1 -- 172.21.15.42:0/1475707713 wait complete. 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96e1110700 1 Processor -- start 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96e1110700 1 -- start start 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96e1110700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 0x7f96dc197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96e1110700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc197ca0 0x7f96dc198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96e1110700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96dc1a07d0 0x7f96dc1a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96e1110700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1a32a0 con 0x7f96dc100050 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96e1110700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1a3410 con 0x7f96dc197ca0 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96e1110700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc1a35d0 con 0x7f96dc1a07d0 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96dad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 0x7f96dc197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96dad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 0x7f96dc197760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60728/0 (socket says 172.21.15.42:60728) 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96dad9d700 1 -- 172.21.15.42:0/3035315006 learned_addr learned my addr 172.21.15.42:0/3035315006 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:51:28.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96da59c700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc197ca0 0x7f96dc198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.267+0000 7f96db59e700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96dc1a07d0 0x7f96dc1a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96dad9d700 1 -- 172.21.15.42:0/3035315006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96dc1a07d0 msgr2=0x7f96dc1a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96dad9d700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96dc1a07d0 0x7f96dc1a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96dad9d700 1 -- 172.21.15.42:0/3035315006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc197ca0 msgr2=0x7f96dc198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96dad9d700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc197ca0 0x7f96dc198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96dad9d700 1 -- 172.21.15.42:0/3035315006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d00095f0 con 0x7f96dc100050 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96dad9d700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 0x7f96dc197760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96d0004ce0 tx=0x7f96d0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96c3fff700 1 -- 172.21.15.42:0/3035315006 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d001d070 con 0x7f96dc100050 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96c3fff700 1 -- 172.21.15.42:0/3035315006 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96d0005000 con 0x7f96dc100050 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96c3fff700 1 -- 172.21.15.42:0/3035315006 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d0022de0 con 0x7f96dc100050 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96dc1a3850 con 0x7f96dc100050 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.268+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc1a3cf0 con 0x7f96dc100050 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.269+0000 7f96c3fff700 1 -- 172.21.15.42:0/3035315006 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f96d000f460 con 0x7f96dc100050 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.269+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc101530 con 0x7f96dc100050 2023-12-12T09:51:28.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.269+0000 7f96c3fff700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f96c4038d40 0x7f96c403b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:28.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.269+0000 7f96c3fff700 1 -- 172.21.15.42:0/3035315006 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f96d004c960 con 0x7f96dc100050 2023-12-12T09:51:28.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.269+0000 7f96da59c700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f96c4038d40 0x7f96c403b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:28.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.270+0000 7f96da59c700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f96c4038d40 0x7f96c403b200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009ad0 tx=0x7f96cc009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:28.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.274+0000 7f96c3fff700 1 -- 172.21.15.42:0/3035315006 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f96d0020550 con 0x7f96dc100050 2023-12-12T09:51:28.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.480+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f96dc10a0f0 con 0x7f96dc100050 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.480+0000 7f96c3fff700 1 -- 172.21.15.42:0/3035315006 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f96d0027030 con 0x7f96dc100050 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi042.stdout:# minimal ceph.conf for 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:51:28.482 INFO:teuthology.orchestra.run.smithi042.stdout: mon_host = [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] 2023-12-12T09:51:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f96c4038d40 msgr2=0x7f96c403b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f96c4038d40 0x7f96c403b200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f96cc009ad0 tx=0x7f96cc009210 comp rx=0 tx=0).stop 2023-12-12T09:51:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 msgr2=0x7f96dc197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 0x7f96dc197760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f96d0004ce0 tx=0x7f96d0004d10 comp rx=0 tx=0).stop 2023-12-12T09:51:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 shutdown_connections 2023-12-12T09:51:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f96c4038d40 0x7f96c403b200 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f96dc1a07d0 0x7f96dc1a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f96dc100050 0x7f96dc197760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:28.484 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 --2- 172.21.15.42:0/3035315006 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f96dc197ca0 0x7f96dc198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 >> 172.21.15.42:0/3035315006 conn(0x7f96dc0fb900 msgr2=0x7f96dc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 shutdown_connections 2023-12-12T09:51:28.485 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:28.482+0000 7f96e1110700 1 -- 172.21.15.42:0/3035315006 wait complete. 2023-12-12T09:51:28.952 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:28 smithi042 ceph-mon[98894]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:28.952 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:28.952 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:28 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3035315006' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:29.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:28 smithi173 ceph-mon[103308]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:29.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:28 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3035315006' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:29.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:28 smithi111 ceph-mon[102565]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:29.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:51:29.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:28 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3035315006' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:29.315 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-12-12T09:51:29.316 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:51:29.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.conf 2023-12-12T09:51:29.352 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:51:29.352 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:51:29.421 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:51:29.421 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.conf 2023-12-12T09:51:29.455 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:51:29.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:51:29.523 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:51:29.523 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.conf 2023-12-12T09:51:29.560 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:51:29.560 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:51:29.634 INFO:tasks.cephadm:Adding mgr.x on smithi042 2023-12-12T09:51:29.634 INFO:tasks.cephadm:Adding mgr.y on smithi111 2023-12-12T09:51:29.634 INFO:tasks.cephadm:Adding mgr.z on smithi173 2023-12-12T09:51:29.634 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch apply mgr '3;smithi042=x;smithi111=y;smithi173=z' 2023-12-12T09:51:30.265 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:51:30.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:30.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:30.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:30 smithi173 ceph-mon[103308]: Reconfiguring mon.b (monmap changed)... 2023-12-12T09:51:30.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:30.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-12T09:51:30.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:30.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:30 smithi173 ceph-mon[103308]: Reconfiguring daemon mon.b on smithi111 2023-12-12T09:51:30.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:30.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:30.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:30 smithi042 ceph-mon[98894]: Reconfiguring mon.b (monmap changed)... 2023-12-12T09:51:30.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:30.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-12T09:51:30.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:30.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:30 smithi042 ceph-mon[98894]: Reconfiguring daemon mon.b on smithi111 2023-12-12T09:51:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:30 smithi111 ceph-mon[102565]: Reconfiguring mon.b (monmap changed)... 2023-12-12T09:51:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-12T09:51:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:30 smithi111 ceph-mon[102565]: Reconfiguring daemon mon.b on smithi111 2023-12-12T09:51:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:31 smithi111 ceph-mon[102565]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:31 smithi111 ceph-mon[102565]: Reconfiguring mon.c (monmap changed)... 2023-12-12T09:51:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-12T09:51:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:31 smithi111 ceph-mon[102565]: Reconfiguring daemon mon.c on smithi173 2023-12-12T09:51:31.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:31 smithi173 ceph-mon[103308]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:31 smithi173 ceph-mon[103308]: Reconfiguring mon.c (monmap changed)... 2023-12-12T09:51:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-12T09:51:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:31 smithi173 ceph-mon[103308]: Reconfiguring daemon mon.c on smithi173 2023-12-12T09:51:31.576 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.575+0000 7f3f01d96700 1 -- 172.21.15.173:0/1800668735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3efc101080 msgr2=0x7f3efc1014a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:31.577 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.575+0000 7f3f01d96700 1 --2- 172.21.15.173:0/1800668735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3efc101080 0x7f3efc1014a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3eec009910 tx=0x7f3eec009c20 comp rx=0 tx=0).stop 2023-12-12T09:51:31.577 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.575+0000 7f3f01d96700 1 -- 172.21.15.173:0/1800668735 shutdown_connections 2023-12-12T09:51:31.577 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.575+0000 7f3f01d96700 1 --2- 172.21.15.173:0/1800668735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3efc101080 0x7f3efc1014a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:31.577 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.575+0000 7f3f01d96700 1 -- 172.21.15.173:0/1800668735 >> 172.21.15.173:0/1800668735 conn(0x7f3efc0fc620 msgr2=0x7f3efc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:31.577 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.575+0000 7f3f01d96700 1 -- 172.21.15.173:0/1800668735 shutdown_connections 2023-12-12T09:51:31.577 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.575+0000 7f3f01d96700 1 -- 172.21.15.173:0/1800668735 wait complete. 2023-12-12T09:51:31.579 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.575+0000 7f3f01d96700 1 Processor -- start 2023-12-12T09:51:31.579 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3f01d96700 1 -- start start 2023-12-12T09:51:31.579 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3f01d96700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3efc101080 0x7f3efc075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:31.579 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3f01d96700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3efc0739c0 0x7f3efc073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:31.579 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3f01d96700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3efc074380 0x7f3efc1a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:31.579 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3f01d96700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc1a0b30 con 0x7f3efc101080 2023-12-12T09:51:31.579 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3f01d96700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc1a0c70 con 0x7f3efc0739c0 2023-12-12T09:51:31.579 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3f01d96700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3efc1a0db0 con 0x7f3efc074380 2023-12-12T09:51:31.579 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3efbfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3efc074380 0x7f3efc1a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3efbfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3efc074380 0x7f3efc1a05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42482/0 (socket says 172.21.15.173:42482) 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3efbfff700 1 -- 172.21.15.173:0/209859400 learned_addr learned my addr 172.21.15.173:0/209859400 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3efb7fe700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3efc101080 0x7f3efc075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3efaffd700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3efc0739c0 0x7f3efc073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3efbfff700 1 -- 172.21.15.173:0/209859400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3efc0739c0 msgr2=0x7f3efc073e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3efbfff700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3efc0739c0 0x7f3efc073e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3efbfff700 1 -- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3efc101080 msgr2=0x7f3efc075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.576+0000 7f3efbfff700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3efc101080 0x7f3efc075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.577+0000 7f3efbfff700 1 -- 172.21.15.173:0/209859400 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec0095f0 con 0x7f3efc074380 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.577+0000 7f3efb7fe700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3efc101080 0x7f3efc075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.577+0000 7f3efbfff700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3efc074380 0x7f3efc1a05f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400ba50 tx=0x7f3ef400be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.577+0000 7f3ef8ff9700 1 -- 172.21.15.173:0/209859400 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400cab0 con 0x7f3efc074380 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.577+0000 7f3ef8ff9700 1 -- 172.21.15.173:0/209859400 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ef400cc10 con 0x7f3efc074380 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.577+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3efc1a0fe0 con 0x7f3efc074380 2023-12-12T09:51:31.580 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.577+0000 7f3ef8ff9700 1 -- 172.21.15.173:0/209859400 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40075b0 con 0x7f3efc074380 2023-12-12T09:51:31.581 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.577+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3efc1a1580 con 0x7f3efc074380 2023-12-12T09:51:31.581 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.578+0000 7f3ef8ff9700 1 -- 172.21.15.173:0/209859400 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7f3ef4007710 con 0x7f3efc074380 2023-12-12T09:51:31.581 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.578+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3efc18fd50 con 0x7f3efc074380 2023-12-12T09:51:31.582 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.578+0000 7f3ef8ff9700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee4038d40 0x7f3ee403b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:31.582 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.578+0000 7f3ef8ff9700 1 -- 172.21.15.173:0/209859400 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3ef404ac30 con 0x7f3efc074380 2023-12-12T09:51:31.582 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.579+0000 7f3efb7fe700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee4038d40 0x7f3ee403b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:31.582 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.579+0000 7f3efb7fe700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee4038d40 0x7f3ee403b200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3eec005130 tx=0x7f3eec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:31.585 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.583+0000 7f3ef8ff9700 1 -- 172.21.15.173:0/209859400 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3ef401b8b0 con 0x7f3efc074380 2023-12-12T09:51:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:31 smithi042 ceph-mon[98894]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:31 smithi042 ceph-mon[98894]: Reconfiguring mon.c (monmap changed)... 2023-12-12T09:51:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-12T09:51:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-12T09:51:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:31 smithi042 ceph-mon[98894]: Reconfiguring daemon mon.c on smithi173 2023-12-12T09:51:31.787 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.785+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi111=y;smithi173=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f3efc02d450 con 0x7f3ee4038d40 2023-12-12T09:51:31.794 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.793+0000 7f3ef8ff9700 1 -- 172.21.15.173:0/209859400 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3efc02d450 con 0x7f3ee4038d40 2023-12-12T09:51:31.794 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mgr update... 2023-12-12T09:51:31.796 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee4038d40 msgr2=0x7f3ee403b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee4038d40 0x7f3ee403b200 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3eec005130 tx=0x7f3eec005fb0 comp rx=0 tx=0).stop 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3efc074380 msgr2=0x7f3efc1a05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3efc074380 0x7f3efc1a05f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400ba50 tx=0x7f3ef400be10 comp rx=0 tx=0).stop 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 shutdown_connections 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3ee4038d40 0x7f3ee403b200 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3efc074380 0x7f3efc1a05f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3efc101080 0x7f3efc075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 --2- 172.21.15.173:0/209859400 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3efc0739c0 0x7f3efc073e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 >> 172.21.15.173:0/209859400 conn(0x7f3efc0fc620 msgr2=0x7f3efc0fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 shutdown_connections 2023-12-12T09:51:31.797 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:51:31.795+0000 7f3f01d96700 1 -- 172.21.15.173:0/209859400 wait complete. 2023-12-12T09:51:32.224 DEBUG:teuthology.orchestra.run.smithi111:mgr.y> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.y.service 2023-12-12T09:51:32.226 DEBUG:teuthology.orchestra.run.smithi173:mgr.z> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.z.service 2023-12-12T09:51:32.229 INFO:tasks.cephadm:Deploying OSDs... 2023-12-12T09:51:32.229 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:51:32.229 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2023-12-12T09:51:32.246 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-12T09:51:32.247 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2023-12-12T09:51:32.258 INFO:journalctl@ceph.mgr.z.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:51:32.260 INFO:journalctl@ceph.mgr.y.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:51:32.304 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-12T09:51:32.304 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:32.304 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 391324 Links: 1 2023-12-12T09:51:32.304 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:32.304 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:32.304 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2023-12-12 09:50:39.469271811 +0000 2023-12-12T09:51:32.304 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2023-12-12 09:44:57.428835641 +0000 2023-12-12T09:51:32.304 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2023-12-12 09:44:57.428835641 +0000 2023-12-12T09:51:32.304 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2023-12-12T09:51:32.304 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-12T09:51:32.370 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2023-12-12T09:51:32.370 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2023-12-12T09:51:32.370 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000113708 s, 4.5 MB/s 2023-12-12T09:51:32.371 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-12T09:51:32.429 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2023-12-12T09:51:32.487 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-12T09:51:32.487 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:32.487 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 394248 Links: 1 2023-12-12T09:51:32.487 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:32.487 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:32.487 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2023-12-12 09:50:39.535271300 +0000 2023-12-12T09:51:32.487 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2023-12-12 09:44:57.959831505 +0000 2023-12-12T09:51:32.487 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2023-12-12 09:44:57.959831505 +0000 2023-12-12T09:51:32.487 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2023-12-12T09:51:32.488 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-12T09:51:32.554 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2023-12-12T09:51:32.554 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2023-12-12T09:51:32.554 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000103899 s, 4.9 MB/s 2023-12-12T09:51:32.555 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-12T09:51:32.613 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2023-12-12T09:51:32.672 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-12T09:51:32.672 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:32.672 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 394414 Links: 1 2023-12-12T09:51:32.672 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:32.672 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:32.672 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2023-12-12 09:50:39.610270718 +0000 2023-12-12T09:51:32.672 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2023-12-12 09:44:58.468827540 +0000 2023-12-12T09:51:32.672 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2023-12-12 09:44:58.468827540 +0000 2023-12-12T09:51:32.673 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2023-12-12T09:51:32.673 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-12T09:51:32.738 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2023-12-12T09:51:32.738 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2023-12-12T09:51:32.738 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000114219 s, 4.5 MB/s 2023-12-12T09:51:32.739 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-12T09:51:32.798 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2023-12-12T09:51:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-12T09:51:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:32.856 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 396361 Links: 1 2023-12-12T09:51:32.856 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:32.856 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:32.856 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2023-12-12 09:50:39.668270269 +0000 2023-12-12T09:51:32.856 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2023-12-12 09:44:58.976823583 +0000 2023-12-12T09:51:32.856 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2023-12-12 09:44:58.976823583 +0000 2023-12-12T09:51:32.856 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: - 2023-12-12T09:51:32.857 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-12T09:51:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2023-12-12T09:51:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2023-12-12T09:51:32.922 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.00010971 s, 4.7 MB/s 2023-12-12T09:51:32.923 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-12T09:51:32.982 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-12-12T09:51:32.982 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2023-12-12T09:51:33.002 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-12T09:51:33.003 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2023-12-12T09:51:33.063 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-12T09:51:33.064 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:33.064 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 391676 Links: 1 2023-12-12T09:51:33.064 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:33.064 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:33.064 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-12-12 09:51:19.387043910 +0000 2023-12-12T09:51:33.064 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-12-12 09:45:05.093703788 +0000 2023-12-12T09:51:33.064 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-12-12 09:45:05.093703788 +0000 2023-12-12T09:51:33.064 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2023-12-12T09:51:33.064 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-12T09:51:33.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:33.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi111=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:33.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: Saving service mgr spec with placement smithi042=x;smithi111=y;smithi173=z;count:3 2023-12-12T09:51:33.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:32 smithi173 ceph-mon[103308]: Deploying daemon mgr.z on smithi173 2023-12-12T09:51:33.131 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:33.131 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi111=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:33.131 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: Saving service mgr spec with placement smithi042=x;smithi111=y;smithi173=z;count:3 2023-12-12T09:51:33.131 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.131 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.131 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.131 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:33.131 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:33.131 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:33.132 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.132 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-12T09:51:33.132 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-12T09:51:33.132 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-12T09:51:33.132 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:33.132 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:32 smithi111 ceph-mon[102565]: Deploying daemon mgr.z on smithi173 2023-12-12T09:51:33.133 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-12-12T09:51:33.134 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-12-12T09:51:33.134 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000307975 s, 1.7 MB/s 2023-12-12T09:51:33.135 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-12T09:51:33.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:33.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi111=y;smithi173=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:33.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: Saving service mgr spec with placement smithi042=x;smithi111=y;smithi173=z;count:3 2023-12-12T09:51:33.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:33.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-12T09:51:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-12T09:51:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-12T09:51:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:32 smithi042 ceph-mon[98894]: Deploying daemon mgr.z on smithi173 2023-12-12T09:51:33.196 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2023-12-12T09:51:33.254 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-12T09:51:33.254 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:33.254 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 391785 Links: 1 2023-12-12T09:51:33.255 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:33.255 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:33.255 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-12-12 09:51:19.387043910 +0000 2023-12-12T09:51:33.255 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-12-12 09:45:05.611694617 +0000 2023-12-12T09:51:33.255 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-12-12 09:45:05.611694617 +0000 2023-12-12T09:51:33.255 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2023-12-12T09:51:33.255 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-12T09:51:33.323 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-12-12T09:51:33.323 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-12-12T09:51:33.323 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000227575 s, 2.2 MB/s 2023-12-12T09:51:33.324 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-12T09:51:33.384 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2023-12-12T09:51:33.443 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-12T09:51:33.444 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 387575 Links: 1 2023-12-12T09:51:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-12-12 09:51:19.387043910 +0000 2023-12-12T09:51:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-12-12 09:45:06.118685641 +0000 2023-12-12T09:51:33.444 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-12-12 09:45:06.118685641 +0000 2023-12-12T09:51:33.444 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2023-12-12T09:51:33.444 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-12T09:51:33.511 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-12-12T09:51:33.511 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-12-12T09:51:33.511 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000230357 s, 2.2 MB/s 2023-12-12T09:51:33.512 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-12T09:51:33.572 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2023-12-12T09:51:33.629 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-12T09:51:33.629 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:33.629 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 6h/6d Inode: 389859 Links: 1 2023-12-12T09:51:33.630 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:33.630 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:33.630 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-12-12 09:51:19.387043910 +0000 2023-12-12T09:51:33.630 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-12-12 09:45:06.642676365 +0000 2023-12-12T09:51:33.630 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-12-12 09:45:06.642676365 +0000 2023-12-12T09:51:33.630 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: - 2023-12-12T09:51:33.630 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-12T09:51:33.696 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-12-12T09:51:33.696 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-12-12T09:51:33.696 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000220033 s, 2.3 MB/s 2023-12-12T09:51:33.697 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-12T09:51:33.757 DEBUG:teuthology.orchestra.run.smithi173:> set -ex 2023-12-12T09:51:33.757 DEBUG:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2023-12-12T09:51:33.773 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-12T09:51:33.773 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 2023-12-12T09:51:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-12T09:51:33.831 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:33.831 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 392497 Links: 1 2023-12-12T09:51:33.831 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:33.831 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:33.831 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2023-12-12 09:51:11.128375188 +0000 2023-12-12T09:51:33.831 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2023-12-12 09:44:45.885643718 +0000 2023-12-12T09:51:33.832 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2023-12-12 09:44:45.885643718 +0000 2023-12-12T09:51:33.832 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2023-12-12T09:51:33.832 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-12T09:51:33.902 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2023-12-12T09:51:33.902 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2023-12-12T09:51:33.902 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000272888 s, 1.9 MB/s 2023-12-12T09:51:33.903 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-12T09:51:33.963 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 2023-12-12T09:51:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-12T09:51:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:34.021 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 394324 Links: 1 2023-12-12T09:51:34.021 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:34.021 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:34.021 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2023-12-12 09:51:33.959889436 +0000 2023-12-12T09:51:34.021 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2023-12-12 09:44:46.443631520 +0000 2023-12-12T09:51:34.021 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2023-12-12 09:44:46.443631520 +0000 2023-12-12T09:51:34.021 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2023-12-12T09:51:34.022 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-12T09:51:34.072 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:34 smithi173 systemd[1]: Starting Ceph mgr.z for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:51:34.094 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2023-12-12T09:51:34.095 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2023-12-12T09:51:34.095 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000226366 s, 2.3 MB/s 2023-12-12T09:51:34.096 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-12T09:51:34.156 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 2023-12-12T09:51:34.214 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-12T09:51:34.214 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:34.214 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 392748 Links: 1 2023-12-12T09:51:34.214 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:34.214 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:34.215 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2023-12-12 09:51:33.959889436 +0000 2023-12-12T09:51:34.215 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2023-12-12 09:44:46.957620283 +0000 2023-12-12T09:51:34.215 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2023-12-12 09:44:46.957620283 +0000 2023-12-12T09:51:34.215 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2023-12-12T09:51:34.215 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-12T09:51:34.279 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2023-12-12T09:51:34.280 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2023-12-12T09:51:34.280 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000202916 s, 2.5 MB/s 2023-12-12T09:51:34.281 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-12T09:51:34.339 DEBUG:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 2023-12-12T09:51:34.398 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-12T09:51:34.398 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-12T09:51:34.398 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 6h/6d Inode: 383936 Links: 1 2023-12-12T09:51:34.398 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:51:34.398 INFO:teuthology.orchestra.run.smithi173.stdout:Context: system_u:object_r:device_t:s0 2023-12-12T09:51:34.398 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2023-12-12 09:51:33.959889436 +0000 2023-12-12T09:51:34.398 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2023-12-12 09:44:47.451609484 +0000 2023-12-12T09:51:34.398 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2023-12-12 09:44:47.451609484 +0000 2023-12-12T09:51:34.398 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2023-12-12T09:51:34.399 DEBUG:teuthology.orchestra.run.smithi173:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-12T09:51:34.463 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:34 smithi173 podman[105280]: 2023-12-12T09:51:34.465 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2023-12-12T09:51:34.466 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2023-12-12T09:51:34.466 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000192507 s, 2.7 MB/s 2023-12-12T09:51:34.467 DEBUG:teuthology.orchestra.run.smithi173:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-12T09:51:34.526 INFO:tasks.cephadm:Deploying osd.0 on smithi042 with /dev/vg_nvme/lv_4... 2023-12-12T09:51:34.546 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-12T09:51:34.741 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:34 smithi173 bash[105280]: f032967940773f81833b0da67bdf1dcb8a57e984665e9c0127703e136a0a5900 2023-12-12T09:51:34.742 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:34 smithi173 ceph-mgr[105444]: -- 172.21.15.173:0/3820804762 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f53ab8d520 con 0x55f53ab62800 2023-12-12T09:51:34.742 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:34 smithi173 systemd[1]: Started Ceph mgr.z for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:51:35.007 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:34 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:34.740+0000 7f9b5da92200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:51:35.007 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:34 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:34.824+0000 7f9b5da92200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:51:35.007 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:34 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:34.912+0000 7f9b5da92200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:51:35.162 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:51:35.322 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:35.005+0000 7f9b5da92200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:51:35.323 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:35.174+0000 7f9b5da92200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:51:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-mon[103308]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-12T09:51:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-12T09:51:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-12T09:51:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:35.450 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:35 smithi111 ceph-mon[102565]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:35.450 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:35.450 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:35.451 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:35.451 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-12T09:51:35.451 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-12T09:51:35.451 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-12T09:51:35.451 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:35.525 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:35 smithi042 ceph-mon[98894]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:35.525 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:35.525 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:35.525 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:35.525 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-12T09:51:35.525 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-12T09:51:35.525 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-12T09:51:35.525 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:35.822 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:35.538+0000 7f9b5da92200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:51:35.822 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:35.672+0000 7f9b5da92200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:51:35.822 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:35.759+0000 7f9b5da92200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:51:36.198 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:35.852+0000 7f9b5da92200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:51:36.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:36 smithi042 ceph-mon[98894]: Deploying daemon mgr.y on smithi111 2023-12-12T09:51:36.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:36 smithi111 ceph-mon[102565]: Deploying daemon mgr.y on smithi111 2023-12-12T09:51:36.497 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:36 smithi111 systemd[1]: Starting Ceph mgr.y for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:51:36.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:36 smithi173 ceph-mon[103308]: Deploying daemon mgr.y on smithi111 2023-12-12T09:51:36.572 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:36 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:36.281+0000 7f9b5da92200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:51:36.682 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-12T09:51:36.682 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-12T09:51:36.682 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2023-12-12T09:51:36.682 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2023-12-12T09:51:36.682 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0082501 s, 1.3 GB/s 2023-12-12T09:51:36.682 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2023-12-12T09:51:36.800 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:36 smithi111 podman[104005]: 2023-12-12T09:51:36.800 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:36 smithi111 bash[104005]: 675e2f66650d47b24a1f381d525b9da3b70ade9ca2839b5141c670d9725f580a 2023-12-12T09:51:36.800 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:36 smithi111 ceph-mgr[104044]: -- 172.21.15.111:0/3147036486 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564c812dd520 con 0x564c812b3000 2023-12-12T09:51:37.124 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_4 2023-12-12T09:51:37.130 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:36 smithi111 systemd[1]: Started Ceph mgr.y for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:51:37.131 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:36 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:36.855+0000 7f6af2809200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:51:37.131 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:36 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:36.936+0000 7f6af2809200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:51:37.131 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:37.029+0000 7f6af2809200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:51:37.131 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:37.128+0000 7f6af2809200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:51:37.322 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:36 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:36.949+0000 7f9b5da92200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:51:37.323 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:37.036+0000 7f9b5da92200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:51:37.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:37 smithi173 ceph-mon[103308]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:37.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:37 smithi042 ceph-mon[98894]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:37.497 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:37.312+0000 7f6af2809200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:51:37.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-mon[102565]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:37.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:37.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:37.727 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:51:37.997 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:37.709+0000 7f6af2809200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:51:37.997 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:37.850+0000 7f6af2809200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:51:37.997 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:37.940+0000 7f6af2809200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:51:38.127 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:37.852+0000 7f9b5da92200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:51:38.127 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:37.942+0000 7f9b5da92200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:51:38.493 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:38.041+0000 7f6af2809200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:51:38.565 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:38.125+0000 7f9b5da92200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:51:38.747 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:38.491+0000 7f6af2809200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:51:38.823 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:38.563+0000 7f9b5da92200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:51:38.823 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:38.652+0000 7f9b5da92200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:51:38.823 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:38.803+0000 7f9b5da92200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:51:38.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:38.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:38.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:38.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:38.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:38.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:38.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:38.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:38.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:38.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-12T09:51:38.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-12T09:51:38.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.950+0000 7fcf93187700 1 -- 172.21.15.42:0/596680070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c101710 msgr2=0x7fcf8c101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.950+0000 7fcf93187700 1 --2- 172.21.15.42:0/596680070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c101710 0x7fcf8c101b70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcf780099c0 tx=0x7fcf78009cd0 comp rx=0 tx=0).stop 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 -- 172.21.15.42:0/596680070 shutdown_connections 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 --2- 172.21.15.42:0/596680070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf8c100510 0x7fcf8c100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 --2- 172.21.15.42:0/596680070 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf8c1020b0 0x7fcf8c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 --2- 172.21.15.42:0/596680070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c101710 0x7fcf8c101b70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 -- 172.21.15.42:0/596680070 >> 172.21.15.42:0/596680070 conn(0x7fcf8c0fbad0 msgr2=0x7fcf8c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 -- 172.21.15.42:0/596680070 shutdown_connections 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 -- 172.21.15.42:0/596680070 wait complete. 2023-12-12T09:51:38.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 Processor -- start 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 -- start start 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf8c100510 0x7fcf8c104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf8c101710 0x7fcf8c102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.951+0000 7fcf93187700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c1020b0 0x7fcf8c103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf93187700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c1037d0 con 0x7fcf8c1020b0 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf93187700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c103940 con 0x7fcf8c101710 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf93187700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c1a15d0 con 0x7fcf8c100510 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf91724700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c1020b0 0x7fcf8c103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf90f23700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf8c100510 0x7fcf8c104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf90f23700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf8c100510 0x7fcf8c104450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:47494/0 (socket says 172.21.15.42:47494) 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf90f23700 1 -- 172.21.15.42:0/2637209640 learned_addr learned my addr 172.21.15.42:0/2637209640 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf91724700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c1020b0 0x7fcf8c103030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60746/0 (socket says 172.21.15.42:60746) 2023-12-12T09:51:38.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf8bfff700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf8c101710 0x7fcf8c102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf90f23700 1 -- 172.21.15.42:0/2637209640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf8c101710 msgr2=0x7fcf8c102af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf90f23700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf8c101710 0x7fcf8c102af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf90f23700 1 -- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c1020b0 msgr2=0x7fcf8c103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf90f23700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c1020b0 0x7fcf8c103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf90f23700 1 -- 172.21.15.42:0/2637209640 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf780095f0 con 0x7fcf8c100510 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf91724700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c1020b0 0x7fcf8c103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf90f23700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf8c100510 0x7fcf8c104450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000e970 tx=0x7fcf8000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.952+0000 7fcf89ffb700 1 -- 172.21.15.42:0/2637209640 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80009790 con 0x7fcf8c100510 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.953+0000 7fcf89ffb700 1 -- 172.21.15.42:0/2637209640 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf8000c7e0 con 0x7fcf8c100510 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.953+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8c1a1860 con 0x7fcf8c100510 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.953+0000 7fcf89ffb700 1 -- 172.21.15.42:0/2637209640 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80003c70 con 0x7fcf8c100510 2023-12-12T09:51:38.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.953+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8c1a1db0 con 0x7fcf8c100510 2023-12-12T09:51:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.954+0000 7fcf89ffb700 1 -- 172.21.15.42:0/2637209640 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 13) v1 ==== 45636+0+0 (secure 0 0 0) 0x7fcf80018430 con 0x7fcf8c100510 2023-12-12T09:51:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.954+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8c04f290 con 0x7fcf8c100510 2023-12-12T09:51:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.954+0000 7fcf89ffb700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf7c038af0 0x7fcf7c03afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:51:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.954+0000 7fcf89ffb700 1 -- 172.21.15.42:0/2637209640 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcf80014070 con 0x7fcf8c100510 2023-12-12T09:51:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.954+0000 7fcf8bfff700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf7c038af0 0x7fcf7c03afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:51:38.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.954+0000 7fcf8bfff700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf7c038af0 0x7fcf7c03afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcf78003730 tx=0x7fcf7800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:51:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:38.959+0000 7fcf89ffb700 1 -- 172.21.15.42:0/2637209640 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcf80003dd0 con 0x7fcf8c100510 2023-12-12T09:51:39.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-12T09:51:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:39.165 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:39.165 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-12T09:51:39.166 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:39.188 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:39.187+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf8c0fdd00 con 0x7fcf7c038af0 2023-12-12T09:51:39.322 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:39.068+0000 7f9b5da92200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:51:39.496 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:39.163+0000 7f6af2809200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:51:39.497 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:39.244+0000 7f6af2809200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:51:39.929 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:39 smithi042 ceph-mon[98894]: Reconfiguring mgr.x (unknown last config time)... 2023-12-12T09:51:39.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:39 smithi042 ceph-mon[98894]: Reconfiguring daemon mgr.x on smithi042 2023-12-12T09:51:39.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:39 smithi042 ceph-mon[98894]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:39.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:51:39.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:51:39.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:40.072 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:39.762+0000 7f9b5da92200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:51:40.072 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:39.895+0000 7f9b5da92200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:51:40.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:39 smithi173 ceph-mon[103308]: Reconfiguring mgr.x (unknown last config time)... 2023-12-12T09:51:40.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:39 smithi173 ceph-mon[103308]: Reconfiguring daemon mgr.x on smithi042 2023-12-12T09:51:40.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:39 smithi173 ceph-mon[103308]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:40.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:51:40.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:51:40.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:40.108 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:40.014+0000 7f6af2809200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:51:40.108 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:39 smithi111 ceph-mon[102565]: Reconfiguring mgr.x (unknown last config time)... 2023-12-12T09:51:40.108 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:39 smithi111 ceph-mon[102565]: Reconfiguring daemon mgr.x on smithi042 2023-12-12T09:51:40.109 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:39 smithi111 ceph-mon[102565]: from='client.24121 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:51:40.109 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:51:40.109 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:51:40.109 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:40.427 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:40.091+0000 7f9b5da92200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:51:40.427 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:40.210+0000 7f9b5da92200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:51:40.496 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:40.106+0000 7f6af2809200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:51:40.497 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:40.279+0000 7f6af2809200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:51:40.822 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:40.425+0000 7f9b5da92200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:51:40.983 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:40.713+0000 7f6af2809200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:51:40.983 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:40.802+0000 7f6af2809200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:51:40.983 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:40.950+0000 7f6af2809200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:51:41.247 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:41.211+0000 7f6af2809200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:51:41.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-mon[102565]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:41.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:41.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:41.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:41.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:41.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:41.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:40 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:41.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-mon[103308]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:41.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:41.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:41.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:41.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:41.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:41.283 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:40 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:41.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:40 smithi042 ceph-mon[98894]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:41.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:40 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:41.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:40 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:41.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:40 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:41.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:40 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:41.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:40 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:41.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:40 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:41.572 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:51:41.281+0000 7f9b5da92200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:51:41.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:41.989+0000 7fcf89ffb700 1 -- 172.21.15.42:0/2637209640 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mgrmap(e 14) v1 ==== 90793+0+0 (secure 0 0 0) 0x7fcf8000cc50 con 0x7fcf8c100510 2023-12-12T09:51:42.241 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:41.914+0000 7f6af2809200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:51:42.241 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:42.045+0000 7f6af2809200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:51:42.241 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:42.239+0000 7f6af2809200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:51:42.242 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-mon[102565]: Standby manager daemon z started 2023-12-12T09:51:42.242 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-12T09:51:42.242 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:51:42.242 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-12T09:51:42.242 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:51:42.242 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2387251775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33717319-c6c1-4217-91a8-02927794dbce"}]: dispatch 2023-12-12T09:51:42.242 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2387251775' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33717319-c6c1-4217-91a8-02927794dbce"}]': finished 2023-12-12T09:51:42.242 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-mon[102565]: osdmap e5: 1 total, 0 up, 1 in 2023-12-12T09:51:42.242 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:42.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-mon[103308]: Standby manager daemon z started 2023-12-12T09:51:42.348 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-12T09:51:42.348 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:51:42.348 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-12T09:51:42.348 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:51:42.348 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2387251775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33717319-c6c1-4217-91a8-02927794dbce"}]: dispatch 2023-12-12T09:51:42.348 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2387251775' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33717319-c6c1-4217-91a8-02927794dbce"}]': finished 2023-12-12T09:51:42.348 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-mon[103308]: osdmap e5: 1 total, 0 up, 1 in 2023-12-12T09:51:42.348 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:41 smithi042 ceph-mon[98894]: Standby manager daemon z started 2023-12-12T09:51:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:41 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-12T09:51:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:41 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:51:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:41 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-12T09:51:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:41 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.173:0/3558064160' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:51:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:41 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2387251775' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33717319-c6c1-4217-91a8-02927794dbce"}]: dispatch 2023-12-12T09:51:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:41 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2387251775' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33717319-c6c1-4217-91a8-02927794dbce"}]': finished 2023-12-12T09:51:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:41 smithi042 ceph-mon[98894]: osdmap e5: 1 total, 0 up, 1 in 2023-12-12T09:51:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:42.497 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:42.352+0000 7f6af2809200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:51:42.989 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:42.570+0000 7f6af2809200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:51:43.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:42 smithi111 ceph-mon[102565]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:43.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:42 smithi111 ceph-mon[102565]: mgrmap e14: x(active, since 74s), standbys: z 2023-12-12T09:51:43.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:42 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-12T09:51:43.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:42 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2299601333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:51:43.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:42 smithi173 ceph-mon[103308]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:43.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:42 smithi173 ceph-mon[103308]: mgrmap e14: x(active, since 74s), standbys: z 2023-12-12T09:51:43.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:42 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-12T09:51:43.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:42 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2299601333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:51:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:42 smithi042 ceph-mon[98894]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:42 smithi042 ceph-mon[98894]: mgrmap e14: x(active, since 74s), standbys: z 2023-12-12T09:51:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:42 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-12T09:51:43.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:42 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2299601333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:51:43.747 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:51:43 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:51:43.452+0000 7f6af2809200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:51:44.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:44.199+0000 7fcf89ffb700 1 -- 172.21.15.42:0/2637209640 <== mon.1 v2:172.21.15.173:3300/0 8 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7fcf800173d0 con 0x7fcf8c100510 2023-12-12T09:51:44.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:44 smithi111 ceph-mon[102565]: Standby manager daemon y started 2023-12-12T09:51:44.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:44 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-12T09:51:44.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:44 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:51:44.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:44 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-12T09:51:44.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:44 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:51:44.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:44 smithi173 ceph-mon[103308]: Standby manager daemon y started 2023-12-12T09:51:44.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:44 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-12T09:51:44.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:44 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:51:44.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:44 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-12T09:51:44.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:44 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:51:44.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:44 smithi042 ceph-mon[98894]: Standby manager daemon y started 2023-12-12T09:51:44.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:44 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-12T09:51:44.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:44 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:51:44.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:44 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-12T09:51:44.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:44 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.111:0/4072254580' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:51:45.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:45 smithi111 ceph-mon[102565]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:45.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:45 smithi111 ceph-mon[102565]: mgrmap e15: x(active, since 76s), standbys: z, y 2023-12-12T09:51:45.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:45 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-12T09:51:45.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:45 smithi173 ceph-mon[103308]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:45.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:45 smithi173 ceph-mon[103308]: mgrmap e15: x(active, since 76s), standbys: z, y 2023-12-12T09:51:45.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:45 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-12T09:51:45.645 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:45 smithi042 ceph-mon[98894]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:45.645 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:45 smithi042 ceph-mon[98894]: mgrmap e15: x(active, since 76s), standbys: z, y 2023-12-12T09:51:45.645 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:45 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-12T09:51:47.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:47 smithi042 ceph-mon[98894]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:47.496 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:47 smithi111 ceph-mon[102565]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:47.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:47 smithi173 ceph-mon[103308]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:49.477 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:49 smithi042 ceph-mon[98894]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:49.477 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:49 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-12T09:51:49.477 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:49 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:49.477 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:49 smithi042 ceph-mon[98894]: Deploying daemon osd.0 on smithi042 2023-12-12T09:51:49.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:49 smithi111 ceph-mon[102565]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:49.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:49 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-12T09:51:49.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:49 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:49.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:49 smithi111 ceph-mon[102565]: Deploying daemon osd.0 on smithi042 2023-12-12T09:51:49.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:49 smithi173 ceph-mon[103308]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:49.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:49 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-12T09:51:49.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:49 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:49.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:49 smithi173 ceph-mon[103308]: Deploying daemon osd.0 on smithi042 2023-12-12T09:51:51.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:51 smithi042 ceph-mon[98894]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:51.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:51 smithi111 ceph-mon[102565]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:51.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:51 smithi173 ceph-mon[103308]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:53.471 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:53 smithi042 ceph-mon[98894]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:53.471 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:53 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:53.471 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:53 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:53.471 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:53 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:53.496 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:53 smithi111 ceph-mon[102565]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:53.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:53 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:53.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:53 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:53.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:53 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:53.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:53 smithi173 ceph-mon[103308]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:53.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:53 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:53.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:53 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:53.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:53 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:55.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:55 smithi111 ceph-mon[102565]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:55.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:55.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:55.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:55.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:55.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:55.513 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:55 smithi042 ceph-mon[98894]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:55.513 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:55.513 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:55.513 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:55.513 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:55.513 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:55.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:55 smithi173 ceph-mon[103308]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:55.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:55.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:55.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:55.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:55.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:55.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.896+0000 7fcf89ffb700 1 -- 172.21.15.42:0/2637209640 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcf8c0fdd00 con 0x7fcf7c038af0 2023-12-12T09:51:55.898 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 0 on host 'smithi042' 2023-12-12T09:51:55.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf7c038af0 msgr2=0x7fcf7c03afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:55.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf7c038af0 0x7fcf7c03afb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcf78003730 tx=0x7fcf7800b540 comp rx=0 tx=0).stop 2023-12-12T09:51:55.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf8c100510 msgr2=0x7fcf8c104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:51:55.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf8c100510 0x7fcf8c104450 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fcf8000e970 tx=0x7fcf8000ec80 comp rx=0 tx=0).stop 2023-12-12T09:51:55.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 shutdown_connections 2023-12-12T09:51:55.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf7c038af0 0x7fcf7c03afb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:55.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf8c1020b0 0x7fcf8c103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:55.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf8c100510 0x7fcf8c104450 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:55.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 --2- 172.21.15.42:0/2637209640 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf8c101710 0x7fcf8c102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:51:55.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 >> 172.21.15.42:0/2637209640 conn(0x7fcf8c0fbad0 msgr2=0x7fcf8c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:51:55.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 shutdown_connections 2023-12-12T09:51:55.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:51:55.898+0000 7fcf93187700 1 -- 172.21.15.42:0/2637209640 wait complete. 2023-12-12T09:51:56.320 DEBUG:teuthology.orchestra.run.smithi042:osd.0> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.0.service 2023-12-12T09:51:56.322 INFO:tasks.cephadm:Deploying osd.1 on smithi111 with /dev/vg_nvme/lv_4... 2023-12-12T09:51:56.322 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-12T09:51:56.354 INFO:journalctl@ceph.osd.0.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:51:56.924 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T09:51:56.998 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:51:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-0[118191]: 2023-12-12T09:51:56.710+0000 7f6f3aebe640 -1 osd.0 0 log_to_monitors true 2023-12-12T09:51:56.998 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:56 smithi042 ceph-mon[98894]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:56.998 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:56 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:56.998 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:56 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:56.998 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:56 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:56.998 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:56 smithi042 ceph-mon[98894]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-12T09:51:57.165 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:56 smithi111 ceph-mon[102565]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:57.165 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:56 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:57.165 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:56 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:57.165 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:56 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:57.165 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:56 smithi111 ceph-mon[102565]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-12T09:51:57.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:56 smithi173 ceph-mon[103308]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:57.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:56 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:57.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:56 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:57.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:56 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:51:57.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:56 smithi173 ceph-mon[103308]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-12T09:51:58.483 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-12T09:51:58.483 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-12T09:51:58.484 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2023-12-12T09:51:58.484 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2023-12-12T09:51:58.484 INFO:teuthology.orchestra.run.smithi111.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142373 s, 736 MB/s 2023-12-12T09:51:58.484 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 2023-12-12T09:51:58.484 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2023-12-12T09:51:58.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:58 smithi111 ceph-mon[102565]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-12T09:51:58.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:58 smithi111 ceph-mon[102565]: osdmap e6: 1 total, 0 up, 1 in 2023-12-12T09:51:58.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:58 smithi111 ceph-mon[102565]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:51:58.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:58 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:58.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:58 smithi173 ceph-mon[103308]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-12T09:51:58.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:58 smithi173 ceph-mon[103308]: osdmap e6: 1 total, 0 up, 1 in 2023-12-12T09:51:58.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:58 smithi173 ceph-mon[103308]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:51:58.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:58 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:58.651 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:51:58 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-0[118191]: 2023-12-12T09:51:58.218+0000 7f6f30535700 -1 osd.0 0 waiting for initial osdmap 2023-12-12T09:51:58.651 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:51:58 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-0[118191]: 2023-12-12T09:51:58.220+0000 7f6f2c32f700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:51:58.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:58 smithi042 ceph-mon[98894]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-12T09:51:58.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:58 smithi042 ceph-mon[98894]: osdmap e6: 1 total, 0 up, 1 in 2023-12-12T09:51:58.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:58 smithi042 ceph-mon[98894]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:51:58.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:58 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:58.872 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi111:vg_nvme/lv_4 2023-12-12T09:51:59.223 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:59.223 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:51:59.223 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: osdmap e7: 1 total, 0 up, 1 in 2023-12-12T09:51:59.223 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:59.223 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:59.223 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.223 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.223 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:51:59.224 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.224 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:59.224 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:59.224 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:51:59 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:59.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:51:59.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: osdmap e7: 1 total, 0 up, 1 in 2023-12-12T09:51:59.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:59.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:59.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:51:59.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:59.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:59.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:51:59 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.463 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T09:51:59.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:51:59.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:51:59.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: osdmap e7: 1 total, 0 up, 1 in 2023-12-12T09:51:59.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:59.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:51:59.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:51:59.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:51:59.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:51:59.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:51:59.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:51:59 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:00.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:00 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:52:00.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:00 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:52:00.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:00 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi042 2023-12-12T09:52:00.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:00 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi042 to 17173M 2023-12-12T09:52:00.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:00 smithi111 ceph-mon[102565]: osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] boot 2023-12-12T09:52:00.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:00 smithi111 ceph-mon[102565]: osdmap e8: 1 total, 1 up, 1 in 2023-12-12T09:52:00.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:00 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:52:00.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:00 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:52:00.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:00 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:52:00.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:00 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi042 2023-12-12T09:52:00.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:00 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi042 to 17173M 2023-12-12T09:52:00.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:00 smithi173 ceph-mon[103308]: osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] boot 2023-12-12T09:52:00.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:00 smithi173 ceph-mon[103308]: osdmap e8: 1 total, 1 up, 1 in 2023-12-12T09:52:00.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:00 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:52:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.644+0000 7fc824546700 1 -- 172.21.15.111:0/2678812386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc81c102820 msgr2=0x7fc81c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.644+0000 7fc824546700 1 --2- 172.21.15.111:0/2678812386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc81c102820 0x7fc81c102c80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc8180099c0 tx=0x7fc818009cd0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.644+0000 7fc824546700 1 -- 172.21.15.111:0/2678812386 shutdown_connections 2023-12-12T09:52:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.644+0000 7fc824546700 1 --2- 172.21.15.111:0/2678812386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc81c1031c0 0x7fc81c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.644+0000 7fc824546700 1 --2- 172.21.15.111:0/2678812386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc81c101620 0x7fc81c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.644+0000 7fc824546700 1 --2- 172.21.15.111:0/2678812386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc81c102820 0x7fc81c102c80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.644+0000 7fc824546700 1 -- 172.21.15.111:0/2678812386 >> 172.21.15.111:0/2678812386 conn(0x7fc81c0fcbe0 msgr2=0x7fc81c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.644+0000 7fc824546700 1 -- 172.21.15.111:0/2678812386 shutdown_connections 2023-12-12T09:52:00.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.644+0000 7fc824546700 1 -- 172.21.15.111:0/2678812386 wait complete. 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc824546700 1 Processor -- start 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc824546700 1 -- start start 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc824546700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc81c101620 0x7fc81c075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc824546700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc81c102820 0x7fc81c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc824546700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc81c1031c0 0x7fc81c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc824546700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0746a0 con 0x7fc81c101620 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc824546700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c074810 con 0x7fc81c102820 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc824546700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c1a14b0 con 0x7fc81c1031c0 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc8222e2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc81c101620 0x7fc81c075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc8222e2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc81c101620 0x7fc81c075320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.111:35578/0 (socket says 172.21.15.111:35578) 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc8222e2700 1 -- 172.21.15.111:0/4243385117 learned_addr learned my addr 172.21.15.111:0/4243385117 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc821ae1700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc81c102820 0x7fc81c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc822ae3700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc81c1031c0 0x7fc81c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc821ae1700 1 -- 172.21.15.111:0/4243385117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc81c1031c0 msgr2=0x7fc81c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc821ae1700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc81c1031c0 0x7fc81c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc821ae1700 1 -- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc81c101620 msgr2=0x7fc81c075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc821ae1700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc81c101620 0x7fc81c075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc821ae1700 1 -- 172.21.15.111:0/4243385117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180095f0 con 0x7fc81c102820 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.645+0000 7fc8222e2700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc81c101620 0x7fc81c075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.646+0000 7fc821ae1700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc81c102820 0x7fc81c0739c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc818009f50 tx=0x7fc8180047a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.646+0000 7fc8137fe700 1 -- 172.21.15.111:0/4243385117 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81801d070 con 0x7fc81c102820 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.646+0000 7fc8137fe700 1 -- 172.21.15.111:0/4243385117 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc818004110 con 0x7fc81c102820 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.646+0000 7fc8137fe700 1 -- 172.21.15.111:0/4243385117 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818022db0 con 0x7fc81c102820 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.646+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c1a16e0 con 0x7fc81c102820 2023-12-12T09:52:00.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.646+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c1a1b80 con 0x7fc81c102820 2023-12-12T09:52:00.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.647+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c1089e0 con 0x7fc81c102820 2023-12-12T09:52:00.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.648+0000 7fc8137fe700 1 -- 172.21.15.111:0/4243385117 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7fc818022470 con 0x7fc81c102820 2023-12-12T09:52:00.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.648+0000 7fc8137fe700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8080a14f0 0x7fc8080a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:00.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.648+0000 7fc8137fe700 1 -- 172.21.15.111:0/4243385117 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fc8180cc8c0 con 0x7fc81c102820 2023-12-12T09:52:00.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.649+0000 7fc8222e2700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8080a14f0 0x7fc8080a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:00.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.649+0000 7fc8222e2700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8080a14f0 0x7fc8080a39b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00f4d0 tx=0x7fc80c00a430 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:00.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:00 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:52:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:00 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:52:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:00 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi042 2023-12-12T09:52:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:00 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi042 to 17173M 2023-12-12T09:52:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:00 smithi042 ceph-mon[98894]: osd.0 [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] boot 2023-12-12T09:52:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:00 smithi042 ceph-mon[98894]: osdmap e8: 1 total, 1 up, 1 in 2023-12-12T09:52:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:00 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:52:00.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.652+0000 7fc8137fe700 1 -- 172.21.15.111:0/4243385117 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc818097980 con 0x7fc81c102820 2023-12-12T09:52:00.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:00.849+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc81c02d380 con 0x7fc8080a14f0 2023-12-12T09:52:01.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:01 smithi111 ceph-mon[102565]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:52:01.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:01 smithi111 ceph-mon[102565]: osdmap e9: 1 total, 1 up, 1 in 2023-12-12T09:52:01.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:52:01.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:52:01.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:01.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:01 smithi173 ceph-mon[103308]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:52:01.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:01 smithi173 ceph-mon[103308]: osdmap e9: 1 total, 1 up, 1 in 2023-12-12T09:52:01.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:52:01.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:52:01.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:01.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:01 smithi042 ceph-mon[98894]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-12T09:52:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:01 smithi042 ceph-mon[98894]: osdmap e9: 1 total, 1 up, 1 in 2023-12-12T09:52:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:52:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:52:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:02.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:02 smithi111 ceph-mon[102565]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:52:02.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:02 smithi173 ceph-mon[103308]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:52:02.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:02 smithi042 ceph-mon[98894]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:52:03.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:03 smithi111 ceph-mon[102565]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:03.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:03 smithi111 ceph-mon[102565]: from='client.? 172.21.15.111:0/3753304619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "067bc3ee-1118-46a9-beb0-cd9e9a237c25"}]: dispatch 2023-12-12T09:52:03.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:03 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "067bc3ee-1118-46a9-beb0-cd9e9a237c25"}]: dispatch 2023-12-12T09:52:03.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:03 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "067bc3ee-1118-46a9-beb0-cd9e9a237c25"}]': finished 2023-12-12T09:52:03.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:03 smithi111 ceph-mon[102565]: osdmap e10: 2 total, 1 up, 2 in 2023-12-12T09:52:03.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:03 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:03.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:03 smithi173 ceph-mon[103308]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:03.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:03 smithi173 ceph-mon[103308]: from='client.? 172.21.15.111:0/3753304619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "067bc3ee-1118-46a9-beb0-cd9e9a237c25"}]: dispatch 2023-12-12T09:52:03.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:03 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "067bc3ee-1118-46a9-beb0-cd9e9a237c25"}]: dispatch 2023-12-12T09:52:03.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:03 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "067bc3ee-1118-46a9-beb0-cd9e9a237c25"}]': finished 2023-12-12T09:52:03.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:03 smithi173 ceph-mon[103308]: osdmap e10: 2 total, 1 up, 2 in 2023-12-12T09:52:03.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:03 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:03 smithi042 ceph-mon[98894]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:03 smithi042 ceph-mon[98894]: from='client.? 172.21.15.111:0/3753304619' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "067bc3ee-1118-46a9-beb0-cd9e9a237c25"}]: dispatch 2023-12-12T09:52:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:03 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "067bc3ee-1118-46a9-beb0-cd9e9a237c25"}]: dispatch 2023-12-12T09:52:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:03 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "067bc3ee-1118-46a9-beb0-cd9e9a237c25"}]': finished 2023-12-12T09:52:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:03 smithi042 ceph-mon[98894]: osdmap e10: 2 total, 1 up, 2 in 2023-12-12T09:52:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:03 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:04.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:04 smithi111 ceph-mon[102565]: from='client.? 172.21.15.111:0/3017579034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:52:04.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:04 smithi173 ceph-mon[103308]: from='client.? 172.21.15.111:0/3017579034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:52:04.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:04 smithi042 ceph-mon[98894]: from='client.? 172.21.15.111:0/3017579034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:52:05.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:05 smithi111 ceph-mon[102565]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:06.045 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:05 smithi042 ceph-mon[98894]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:06.045 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:05 smithi173 ceph-mon[103308]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:07.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:07 smithi111 ceph-mon[102565]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:07.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:07 smithi173 ceph-mon[103308]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:07 smithi042 ceph-mon[98894]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:09.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:09 smithi111 ceph-mon[102565]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:09.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:09 smithi173 ceph-mon[103308]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:09.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:09 smithi042 ceph-mon[98894]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:10.539 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-12T09:52:10.539 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:10.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-12T09:52:10.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:10.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-12T09:52:10.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:11.461 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:11 smithi111 ceph-mon[102565]: Deploying daemon osd.1 on smithi111 2023-12-12T09:52:11.461 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:11 smithi111 ceph-mon[102565]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:11.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:11 smithi173 ceph-mon[103308]: Deploying daemon osd.1 on smithi111 2023-12-12T09:52:11.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:11 smithi173 ceph-mon[103308]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:11.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:11 smithi042 ceph-mon[98894]: Deploying daemon osd.1 on smithi111 2023-12-12T09:52:11.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:11 smithi042 ceph-mon[98894]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:13.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:13 smithi111 ceph-mon[102565]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:13.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:13 smithi173 ceph-mon[103308]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:13.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:13 smithi042 ceph-mon[98894]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:15.172 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:15 smithi111 ceph-mon[102565]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:15.172 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:15.172 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:15.172 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:15.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:15 smithi173 ceph-mon[103308]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:15.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:15.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:15.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:15 smithi042 ceph-mon[98894]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:16.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.765+0000 7fc8137fe700 1 -- 172.21.15.111:0/4243385117 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc81c02d380 con 0x7fc8080a14f0 2023-12-12T09:52:16.767 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 1 on host 'smithi111' 2023-12-12T09:52:16.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8080a14f0 msgr2=0x7fc8080a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8080a14f0 0x7fc8080a39b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00f4d0 tx=0x7fc80c00a430 comp rx=0 tx=0).stop 2023-12-12T09:52:16.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc81c102820 msgr2=0x7fc81c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:16.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc81c102820 0x7fc81c0739c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc818009f50 tx=0x7fc8180047a0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 shutdown_connections 2023-12-12T09:52:16.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8080a14f0 0x7fc8080a39b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc81c1031c0 0x7fc81c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc81c101620 0x7fc81c075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 --2- 172.21.15.111:0/4243385117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc81c102820 0x7fc81c0739c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:16.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 >> 172.21.15.111:0/4243385117 conn(0x7fc81c0fcbe0 msgr2=0x7fc81c105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:16.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 shutdown_connections 2023-12-12T09:52:16.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:52:16.768+0000 7fc824546700 1 -- 172.21.15.111:0/4243385117 wait complete. 2023-12-12T09:52:17.269 DEBUG:teuthology.orchestra.run.smithi111:osd.1> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.1.service 2023-12-12T09:52:17.271 INFO:tasks.cephadm:Deploying osd.2 on smithi173 with /dev/vg_nvme/lv_4... 2023-12-12T09:52:17.271 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-12T09:52:17.306 INFO:journalctl@ceph.osd.1.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:52:17.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:17 smithi173 ceph-mon[103308]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:17.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:17.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:17.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:17 smithi042 ceph-mon[98894]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:17.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-mon[102565]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:17.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:17.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:17.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.678 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:17.679 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:17.870 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:52:18.246 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:17 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-1[107648]: 2023-12-12T09:52:17.985+0000 7f9b0c328640 -1 osd.1 0 log_to_monitors true 2023-12-12T09:52:18.358 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:18 smithi173 ceph-mon[103308]: from='osd.1 [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-12T09:52:18.359 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:18 smithi173 ceph-mon[103308]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-12T09:52:18.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:18 smithi042 ceph-mon[98894]: from='osd.1 [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-12T09:52:18.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:18 smithi042 ceph-mon[98894]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-12T09:52:18.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:18 smithi111 ceph-mon[102565]: from='osd.1 [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-12T09:52:18.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:18 smithi111 ceph-mon[102565]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-12T09:52:19.397 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-12T09:52:19.398 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-12T09:52:19.398 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2023-12-12T09:52:19.398 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2023-12-12T09:52:19.398 INFO:teuthology.orchestra.run.smithi173.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127519 s, 822 MB/s 2023-12-12T09:52:19.398 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 2023-12-12T09:52:19.398 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2023-12-12T09:52:19.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:19 smithi173 ceph-mon[103308]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:19.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:19 smithi173 ceph-mon[103308]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-12T09:52:19.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:19 smithi173 ceph-mon[103308]: osdmap e11: 2 total, 1 up, 2 in 2023-12-12T09:52:19.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:19 smithi173 ceph-mon[103308]: from='osd.1 [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:52:19.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:19 smithi173 ceph-mon[103308]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:52:19.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:19.580 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:19 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-1[107648]: 2023-12-12T09:52:19.289+0000 7f9b0199f700 -1 osd.1 0 waiting for initial osdmap 2023-12-12T09:52:19.580 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:19 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-1[107648]: 2023-12-12T09:52:19.293+0000 7f9afd799700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:52:19.580 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:19 smithi111 ceph-mon[102565]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:19.580 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:19 smithi111 ceph-mon[102565]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-12T09:52:19.580 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:19 smithi111 ceph-mon[102565]: osdmap e11: 2 total, 1 up, 2 in 2023-12-12T09:52:19.580 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:19 smithi111 ceph-mon[102565]: from='osd.1 [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:52:19.580 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:19 smithi111 ceph-mon[102565]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:52:19.581 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:19.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:19 smithi042 ceph-mon[98894]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:19.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:19 smithi042 ceph-mon[98894]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-12T09:52:19.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:19 smithi042 ceph-mon[98894]: osdmap e11: 2 total, 1 up, 2 in 2023-12-12T09:52:19.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:19 smithi042 ceph-mon[98894]: from='osd.1 [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:52:19.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:19 smithi042 ceph-mon[98894]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:52:19.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:19.834 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi173:vg_nvme/lv_4 2023-12-12T09:52:20.427 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:52:20.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:52:20.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: osdmap e12: 2 total, 1 up, 2 in 2023-12-12T09:52:20.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:20.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:20.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:52:20.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:20.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:20.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:20 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: osdmap e12: 2 total, 1 up, 2 in 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:20 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: osdmap e12: 2 total, 1 up, 2 in 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:20 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:21.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:21 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:52:21.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:21 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:52:21.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:21 smithi173 ceph-mon[103308]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:21.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:21 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi111 2023-12-12T09:52:21.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:21 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi111 to 17168M 2023-12-12T09:52:21.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:21.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:21 smithi173 ceph-mon[103308]: osd.1 [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] boot 2023-12-12T09:52:21.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:21 smithi173 ceph-mon[103308]: osdmap e13: 2 total, 2 up, 2 in 2023-12-12T09:52:21.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:21.640 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.635+0000 7f64e0a4f700 1 -- 172.21.15.173:0/3993748529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc101120 msgr2=0x7f64dc1089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:21.640 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.635+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/3993748529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc101120 0x7f64dc1089c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b0a0 tx=0x7f64d400b3b0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.640 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.636+0000 7f64e0a4f700 1 -- 172.21.15.173:0/3993748529 shutdown_connections 2023-12-12T09:52:21.640 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.636+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/3993748529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc101120 0x7f64dc1089c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.640 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.636+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/3993748529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64dc101b90 0x7f64dc101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.640 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.636+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/3993748529 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64dc100760 0x7f64dc100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.636+0000 7f64e0a4f700 1 -- 172.21.15.173:0/3993748529 >> 172.21.15.173:0/3993748529 conn(0x7f64dc0fb230 msgr2=0x7f64dc0fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.636+0000 7f64e0a4f700 1 -- 172.21.15.173:0/3993748529 shutdown_connections 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.636+0000 7f64e0a4f700 1 -- 172.21.15.173:0/3993748529 wait complete. 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.636+0000 7f64e0a4f700 1 Processor -- start 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.636+0000 7f64e0a4f700 1 -- start start 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64e0a4f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc100760 0x7f64dc100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64e0a4f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64dc101120 0x7f64dc19c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64e0a4f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64dc101b90 0x7f64dc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64e0a4f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc19f040 con 0x7f64dc101b90 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64e0a4f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc19f1b0 con 0x7f64dc101120 2023-12-12T09:52:21.641 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64e0a4f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc19f370 con 0x7f64dc100760 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64da59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc100760 0x7f64dc100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64da59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc100760 0x7f64dc100110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42508/0 (socket says 172.21.15.173:42508) 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64da59c700 1 -- 172.21.15.173:0/2144399256 learned_addr learned my addr 172.21.15.173:0/2144399256 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64d9d9b700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64dc101120 0x7f64dc19c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64da59c700 1 -- 172.21.15.173:0/2144399256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64dc101120 msgr2=0x7f64dc19c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64da59c700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64dc101120 0x7f64dc19c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64da59c700 1 -- 172.21.15.173:0/2144399256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64dc101b90 msgr2=0x7f64dc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64da59c700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64dc101b90 0x7f64dc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64da59c700 1 -- 172.21.15.173:0/2144399256 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d4009d00 con 0x7f64dc100760 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64da59c700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc100760 0x7f64dc100110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c9b0 tx=0x7f64c400cd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:21.642 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64d37fe700 1 -- 172.21.15.173:0/2144399256 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4007b00 con 0x7f64dc100760 2023-12-12T09:52:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64d37fe700 1 -- 172.21.15.173:0/2144399256 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64c4007c60 con 0x7f64dc100760 2023-12-12T09:52:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc19f650 con 0x7f64dc100760 2023-12-12T09:52:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.637+0000 7f64d37fe700 1 -- 172.21.15.173:0/2144399256 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4018660 con 0x7f64dc100760 2023-12-12T09:52:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.638+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc19fb50 con 0x7f64dc100760 2023-12-12T09:52:21.643 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.638+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc107930 con 0x7f64dc100760 2023-12-12T09:52:21.644 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.640+0000 7f64d37fe700 1 -- 172.21.15.173:0/2144399256 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7f64c4018880 con 0x7f64dc100760 2023-12-12T09:52:21.644 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.640+0000 7f64d37fe700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f64c80a13f0 0x7f64c80a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:21.644 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.640+0000 7f64d37fe700 1 -- 172.21.15.173:0/2144399256 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f64c40cb7b0 con 0x7f64dc100760 2023-12-12T09:52:21.644 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.640+0000 7f64d9d9b700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f64c80a13f0 0x7f64c80a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:21.644 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.641+0000 7f64d9d9b700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f64c80a13f0 0x7f64c80a38b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64cc005950 tx=0x7f64cc005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:21.645 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.644+0000 7f64d37fe700 1 -- 172.21.15.173:0/2144399256 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f64c40966f0 con 0x7f64dc100760 2023-12-12T09:52:21.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:21 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:52:21.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:21 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:52:21.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:21 smithi042 ceph-mon[98894]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:21.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:21 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi111 2023-12-12T09:52:21.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:21 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi111 to 17168M 2023-12-12T09:52:21.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:21.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:21 smithi042 ceph-mon[98894]: osd.1 [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] boot 2023-12-12T09:52:21.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:21 smithi042 ceph-mon[98894]: osdmap e13: 2 total, 2 up, 2 in 2023-12-12T09:52:21.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:21 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:52:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:21 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:52:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:21 smithi111 ceph-mon[102565]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-12-12T09:52:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:21 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi111 2023-12-12T09:52:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:21 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi111 to 17168M 2023-12-12T09:52:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:21 smithi111 ceph-mon[102565]: osd.1 [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] boot 2023-12-12T09:52:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:21 smithi111 ceph-mon[102565]: osdmap e13: 2 total, 2 up, 2 in 2023-12-12T09:52:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:52:21.843 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:21.841+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f64dc02d380 con 0x7f64c80a13f0 2023-12-12T09:52:22.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:22 smithi173 ceph-mon[103308]: osdmap e14: 2 total, 2 up, 2 in 2023-12-12T09:52:22.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:22 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:52:22.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:22 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:52:22.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:22 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:22.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:22 smithi042 ceph-mon[98894]: osdmap e14: 2 total, 2 up, 2 in 2023-12-12T09:52:22.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:22 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:52:22.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:22 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:52:22.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:22 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:22.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:22 smithi111 ceph-mon[102565]: osdmap e14: 2 total, 2 up, 2 in 2023-12-12T09:52:22.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:22 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:52:22.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:22 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:52:22.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:22 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:23.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:23 smithi173 ceph-mon[103308]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:23.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:23 smithi173 ceph-mon[103308]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:52:23.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:23 smithi042 ceph-mon[98894]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:23.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:23 smithi042 ceph-mon[98894]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:52:23.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:23 smithi111 ceph-mon[102565]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:23.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:23 smithi111 ceph-mon[102565]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:52:24.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:24 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/2596472206' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d209df1-c616-4f0e-8a58-5b0c1839967c"}]: dispatch 2023-12-12T09:52:24.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:24 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d209df1-c616-4f0e-8a58-5b0c1839967c"}]: dispatch 2023-12-12T09:52:24.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:24 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d209df1-c616-4f0e-8a58-5b0c1839967c"}]': finished 2023-12-12T09:52:24.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:24 smithi173 ceph-mon[103308]: osdmap e15: 3 total, 2 up, 3 in 2023-12-12T09:52:24.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:24 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:24.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:24 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/2596472206' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d209df1-c616-4f0e-8a58-5b0c1839967c"}]: dispatch 2023-12-12T09:52:24.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:24 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d209df1-c616-4f0e-8a58-5b0c1839967c"}]: dispatch 2023-12-12T09:52:24.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:24 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d209df1-c616-4f0e-8a58-5b0c1839967c"}]': finished 2023-12-12T09:52:24.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:24 smithi042 ceph-mon[98894]: osdmap e15: 3 total, 2 up, 3 in 2023-12-12T09:52:24.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:24 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:24.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:24 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/2596472206' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d209df1-c616-4f0e-8a58-5b0c1839967c"}]: dispatch 2023-12-12T09:52:24.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:24 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1d209df1-c616-4f0e-8a58-5b0c1839967c"}]: dispatch 2023-12-12T09:52:24.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:24 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1d209df1-c616-4f0e-8a58-5b0c1839967c"}]': finished 2023-12-12T09:52:24.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:24 smithi111 ceph-mon[102565]: osdmap e15: 3 total, 2 up, 3 in 2023-12-12T09:52:24.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:24 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:25.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:25 smithi173 ceph-mon[103308]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:25.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:25 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/3436830505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:52:25.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:25 smithi042 ceph-mon[98894]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:25.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:25 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/3436830505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:52:25.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:25 smithi111 ceph-mon[102565]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:25.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:25 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/3436830505' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:52:27.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:27 smithi173 ceph-mon[103308]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:27 smithi042 ceph-mon[98894]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:27.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:27 smithi111 ceph-mon[102565]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:29.498 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:29 smithi173 ceph-mon[103308]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:29 smithi042 ceph-mon[98894]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:29.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:29 smithi111 ceph-mon[102565]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:31.630 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:31 smithi173 ceph-mon[103308]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:31.630 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-12T09:52:31.630 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:31 smithi042 ceph-mon[98894]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-12T09:52:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:31 smithi111 ceph-mon[102565]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-12T09:52:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:32.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:32 smithi173 ceph-mon[103308]: Deploying daemon osd.2 on smithi173 2023-12-12T09:52:32.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:32 smithi042 ceph-mon[98894]: Deploying daemon osd.2 on smithi173 2023-12-12T09:52:32.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:32 smithi111 ceph-mon[102565]: Deploying daemon osd.2 on smithi173 2023-12-12T09:52:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:33 smithi173 ceph-mon[103308]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:33.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:33 smithi042 ceph-mon[98894]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:33.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:33 smithi111 ceph-mon[102565]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:35.594 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:35 smithi173 ceph-mon[103308]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:35.594 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:35.594 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:35.594 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:35.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:35 smithi042 ceph-mon[98894]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:35.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:35.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:35.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:35 smithi111 ceph-mon[102565]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:37 smithi042 ceph-mon[98894]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:37.664 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:37 smithi173 ceph-mon[103308]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:37 smithi111 ceph-mon[102565]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:37.954 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.952+0000 7f64d37fe700 1 -- 172.21.15.173:0/2144399256 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f64dc02d380 con 0x7f64c80a13f0 2023-12-12T09:52:37.955 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 2 on host 'smithi173' 2023-12-12T09:52:37.957 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f64c80a13f0 msgr2=0x7f64c80a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.957 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f64c80a13f0 0x7f64c80a38b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64cc005950 tx=0x7f64cc005cf0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.957 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc100760 msgr2=0x7f64dc100110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:37.957 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc100760 0x7f64dc100110 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c9b0 tx=0x7f64c400cd70 comp rx=0 tx=0).stop 2023-12-12T09:52:37.957 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 shutdown_connections 2023-12-12T09:52:37.957 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f64c80a13f0 0x7f64c80a38b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.957 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f64dc101b90 0x7f64dc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.957 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64dc100760 0x7f64dc100110 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.957 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 --2- 172.21.15.173:0/2144399256 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64dc101120 0x7f64dc19c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:37.958 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 >> 172.21.15.173:0/2144399256 conn(0x7f64dc0fb230 msgr2=0x7f64dc104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:37.958 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 shutdown_connections 2023-12-12T09:52:37.958 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:52:37.955+0000 7f64e0a4f700 1 -- 172.21.15.173:0/2144399256 wait complete. 2023-12-12T09:52:38.328 DEBUG:teuthology.orchestra.run.smithi173:osd.2> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.2.service 2023-12-12T09:52:38.339 INFO:tasks.cephadm:Deploying osd.3 on smithi042 with /dev/vg_nvme/lv_3... 2023-12-12T09:52:38.340 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-12T09:52:38.373 INFO:journalctl@ceph.osd.2.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:52:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:38.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:38.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:52:39.484 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-2[109160]: 2023-12-12T09:52:39.114+0000 7fc88dfa1640 -1 osd.2 0 log_to_monitors true 2023-12-12T09:52:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:39 smithi111 ceph-mon[102565]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:39 smithi111 ceph-mon[102565]: from='osd.2 [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-12T09:52:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:39 smithi111 ceph-mon[102565]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-12T09:52:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:39 smithi173 ceph-mon[103308]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:39 smithi173 ceph-mon[103308]: from='osd.2 [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-12T09:52:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:39 smithi173 ceph-mon[103308]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-12T09:52:39.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:39 smithi042 ceph-mon[98894]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:39.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:39 smithi042 ceph-mon[98894]: from='osd.2 [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-12T09:52:39.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:39 smithi042 ceph-mon[98894]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-12T09:52:40.538 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-12T09:52:40.538 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-12T09:52:40.539 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2023-12-12T09:52:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2023-12-12T09:52:40.539 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00795242 s, 1.3 GB/s 2023-12-12T09:52:40.539 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2023-12-12T09:52:40.823 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-2[109160]: 2023-12-12T09:52:40.500+0000 7fc883618700 -1 osd.2 0 waiting for initial osdmap 2023-12-12T09:52:40.823 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-2[109160]: 2023-12-12T09:52:40.504+0000 7fc87bc0b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:52:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:40 smithi173 ceph-mon[103308]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-12T09:52:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:40 smithi173 ceph-mon[103308]: osdmap e16: 3 total, 2 up, 3 in 2023-12-12T09:52:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:40 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:40 smithi173 ceph-mon[103308]: from='osd.2 [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:52:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:40 smithi173 ceph-mon[103308]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:52:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:40 smithi042 ceph-mon[98894]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-12T09:52:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:40 smithi042 ceph-mon[98894]: osdmap e16: 3 total, 2 up, 3 in 2023-12-12T09:52:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:40 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:40 smithi042 ceph-mon[98894]: from='osd.2 [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:52:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:40 smithi042 ceph-mon[98894]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:52:40.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:40 smithi111 ceph-mon[102565]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-12T09:52:40.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:40 smithi111 ceph-mon[102565]: osdmap e16: 3 total, 2 up, 3 in 2023-12-12T09:52:40.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:40 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:40.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:40 smithi111 ceph-mon[102565]: from='osd.2 [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:52:40.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:40 smithi111 ceph-mon[102565]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:52:41.030 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_3 2023-12-12T09:52:41.677 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:52:41.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: osdmap e17: 3 total, 2 up, 3 in 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: osdmap e17: 3 total, 2 up, 3 in 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: osdmap e17: 3 total, 2 up, 3 in 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:52:41.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:42 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:52:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:42 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:52:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:42 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi173 2023-12-12T09:52:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:42 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi173 to 17168M 2023-12-12T09:52:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:42 smithi042 ceph-mon[98894]: osd.2 [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] boot 2023-12-12T09:52:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:42 smithi042 ceph-mon[98894]: osdmap e18: 3 total, 3 up, 3 in 2023-12-12T09:52:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:42 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:42 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-12T09:52:42.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:42 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:52:42.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:42 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:52:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:42 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi173 2023-12-12T09:52:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:42 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi173 to 17168M 2023-12-12T09:52:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:42 smithi173 ceph-mon[103308]: osd.2 [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] boot 2023-12-12T09:52:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:42 smithi173 ceph-mon[103308]: osdmap e18: 3 total, 3 up, 3 in 2023-12-12T09:52:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:42 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:42 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-12T09:52:42.985 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 -- 172.21.15.42:0/3470444117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c101620 msgr2=0x7f685c101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 --2- 172.21.15.42:0/3470444117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c101620 0x7f685c101a40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f68580099c0 tx=0x7f6858009cd0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 -- 172.21.15.42:0/3470444117 shutdown_connections 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 --2- 172.21.15.42:0/3470444117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f685c1031c0 0x7f685c109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 --2- 172.21.15.42:0/3470444117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c101620 0x7f685c101a40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 --2- 172.21.15.42:0/3470444117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f685c102820 0x7f685c102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 -- 172.21.15.42:0/3470444117 >> 172.21.15.42:0/3470444117 conn(0x7f685c0fcbe0 msgr2=0x7f685c0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 -- 172.21.15.42:0/3470444117 shutdown_connections 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 -- 172.21.15.42:0/3470444117 wait complete. 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.982+0000 7f6863549700 1 Processor -- start 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6863549700 1 -- start start 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6863549700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f685c101620 0x7f685c075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.986 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6863549700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f685c102820 0x7f685c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6863549700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c1031c0 0x7f685c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6863549700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c0746a0 con 0x7f685c1031c0 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6863549700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c074810 con 0x7f685c102820 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6863549700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685c1a1560 con 0x7f685c101620 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6861ae6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c1031c0 0x7f685c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6861ae6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c1031c0 0x7f685c073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60816/0 (socket says 172.21.15.42:60816) 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6861ae6700 1 -- 172.21.15.42:0/3459170000 learned_addr learned my addr 172.21.15.42:0/3459170000 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f68612e5700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f685c101620 0x7f685c075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6860ae4700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f685c102820 0x7f685c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6861ae6700 1 -- 172.21.15.42:0/3459170000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f685c101620 msgr2=0x7f685c075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6861ae6700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f685c101620 0x7f685c075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6861ae6700 1 -- 172.21.15.42:0/3459170000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f685c102820 msgr2=0x7f685c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6861ae6700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f685c102820 0x7f685c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6861ae6700 1 -- 172.21.15.42:0/3459170000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68580095f0 con 0x7f685c1031c0 2023-12-12T09:52:42.987 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6860ae4700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f685c102820 0x7f685c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:52:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.983+0000 7f6861ae6700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c1031c0 0x7f685c073f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f685400ada0 tx=0x7f685400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.984+0000 7f68527fc700 1 -- 172.21.15.42:0/3459170000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685400f070 con 0x7f685c1031c0 2023-12-12T09:52:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.984+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685c1a17f0 con 0x7f685c1031c0 2023-12-12T09:52:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.984+0000 7f68527fc700 1 -- 172.21.15.42:0/3459170000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6854007b30 con 0x7f685c1031c0 2023-12-12T09:52:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.984+0000 7f68527fc700 1 -- 172.21.15.42:0/3459170000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68540053e0 con 0x7f685c1031c0 2023-12-12T09:52:42.988 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.984+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685c1a1cf0 con 0x7f685c1031c0 2023-12-12T09:52:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.985+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685c108b00 con 0x7f685c1031c0 2023-12-12T09:52:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.986+0000 7f68527fc700 1 -- 172.21.15.42:0/3459170000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 15) v1 ==== 135950+0+0 (secure 0 0 0) 0x7f685400c980 con 0x7f685c1031c0 2023-12-12T09:52:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.986+0000 7f68527fc700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f68480a13f0 0x7f68480a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:52:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.986+0000 7f68612e5700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f68480a13f0 0x7f68480a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:52:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.986+0000 7f68527fc700 1 -- 172.21.15.42:0/3459170000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7f68540cb170 con 0x7f685c1031c0 2023-12-12T09:52:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.987+0000 7f68612e5700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f68480a13f0 0x7f68480a38b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6858003820 tx=0x7f6858005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:52:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:42.990+0000 7f68527fc700 1 -- 172.21.15.42:0/3459170000 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6854095d20 con 0x7f685c1031c0 2023-12-12T09:52:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:42 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:52:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:42 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:52:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:42 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi173 2023-12-12T09:52:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:42 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi173 to 17168M 2023-12-12T09:52:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:42 smithi111 ceph-mon[102565]: osd.2 [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] boot 2023-12-12T09:52:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:42 smithi111 ceph-mon[102565]: osdmap e18: 3 total, 3 up, 3 in 2023-12-12T09:52:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:42 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:52:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:42 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-12T09:52:43.202 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:43.201+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f685c061a10 con 0x7f68480a13f0 2023-12-12T09:52:43.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:43 smithi173 ceph-mon[103308]: pgmap v75: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-12T09:52:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:43 smithi173 ceph-mon[103308]: osdmap e19: 3 total, 3 up, 3 in 2023-12-12T09:52:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-12T09:52:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:52:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:52:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:43 smithi042 ceph-mon[98894]: pgmap v75: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-12T09:52:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:43 smithi042 ceph-mon[98894]: osdmap e19: 3 total, 3 up, 3 in 2023-12-12T09:52:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-12T09:52:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:52:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:52:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:43.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:43 smithi111 ceph-mon[102565]: pgmap v75: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:43.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-12T09:52:43.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:43 smithi111 ceph-mon[102565]: osdmap e19: 3 total, 3 up, 3 in 2023-12-12T09:52:43.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-12T09:52:43.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:52:43.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:52:43.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:44.372 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:52:44 smithi042 sudo[123895]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-12T09:52:44.373 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:52:44 smithi042 sudo[123895]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:44.373 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:52:44 smithi042 sudo[123895]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:44.373 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:52:44 smithi042 sudo[123895]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:44.530 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:52:44.528+0000 7f68527fc700 1 -- 172.21.15.42:0/3459170000 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f68540960f0 con 0x7f685c1031c0 2023-12-12T09:52:44.652 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:44 smithi042 ceph-mon[98894]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:52:44.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:44 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-12T09:52:44.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:44 smithi042 ceph-mon[98894]: osdmap e20: 3 total, 3 up, 3 in 2023-12-12T09:52:44.653 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:52:44 smithi042 sudo[123931]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-12T09:52:44.653 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:52:44 smithi042 sudo[123931]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:44.653 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:52:44 smithi042 sudo[123931]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:44.654 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 09:52:44 smithi042 sudo[123931]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:44.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:44 smithi111 ceph-mon[102565]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:52:44.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:44 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-12T09:52:44.799 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:44 smithi111 ceph-mon[102565]: osdmap e20: 3 total, 3 up, 3 in 2023-12-12T09:52:44.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:44 smithi173 ceph-mon[103308]: from='client.14295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:52:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:44 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-12T09:52:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:44 smithi173 ceph-mon[103308]: osdmap e20: 3 total, 3 up, 3 in 2023-12-12T09:52:45.053 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:44 smithi111 sudo[113163]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-12T09:52:45.054 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:44 smithi111 sudo[113163]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:45.054 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:44 smithi111 sudo[113163]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:45.054 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:44 smithi111 sudo[113163]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:45.497 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:45 smithi111 sudo[113166]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-12T09:52:45.497 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:45 smithi111 sudo[113166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:45.497 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:45 smithi111 sudo[113166]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:45.497 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 09:52:45 smithi111 sudo[113166]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:45.578 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:45 smithi173 ceph-mon[103308]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:45.578 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:45 smithi173 ceph-mon[103308]: osdmap e21: 3 total, 3 up, 3 in 2023-12-12T09:52:45.578 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:45 smithi173 ceph-mon[103308]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-12-12T09:52:45.579 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:45 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:45.579 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:45 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:45.579 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:45 smithi173 sudo[113830]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-12T09:52:45.579 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:45 smithi173 sudo[113830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:45.579 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:45 smithi173 sudo[113830]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:45.579 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:45 smithi173 sudo[113830]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:45.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:45 smithi042 ceph-mon[98894]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:45.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:45 smithi042 ceph-mon[98894]: osdmap e21: 3 total, 3 up, 3 in 2023-12-12T09:52:45.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:45 smithi042 ceph-mon[98894]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-12-12T09:52:45.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:45 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:45.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:45 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:45.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:45 smithi111 ceph-mon[102565]: pgmap v78: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:45.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:45 smithi111 ceph-mon[102565]: osdmap e21: 3 total, 3 up, 3 in 2023-12-12T09:52:45.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:45 smithi111 ceph-mon[102565]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-12-12T09:52:45.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:45 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:45.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:45 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:46.072 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:45 smithi173 sudo[113833]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-12T09:52:46.072 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:45 smithi173 sudo[113833]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:46.073 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:45 smithi173 sudo[113833]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:46.073 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 09:52:45 smithi173 sudo[113833]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:46.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:46 smithi042 sudo[124028]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-12T09:52:46.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:46 smithi042 sudo[124028]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:46.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:46 smithi042 sudo[124028]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:46.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:46 smithi042 sudo[124028]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:46 smithi042 sudo[124031]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-12T09:52:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:46 smithi042 sudo[124031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:46 smithi042 sudo[124031]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:46 smithi042 sudo[124031]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:46.995 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:46 smithi173 sudo[113836]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-12T09:52:46.995 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:46 smithi173 sudo[113836]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:46.995 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:46 smithi173 sudo[113836]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:46.995 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:46 smithi173 sudo[113836]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:47.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:46 smithi173 sudo[113839]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-12T09:52:47.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:46 smithi173 sudo[113839]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:47.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:46 smithi173 sudo[113839]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:47.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:46 smithi173 sudo[113839]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:47.489 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 sudo[113169]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-12T09:52:47.489 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 sudo[113169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:47.489 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 sudo[113169]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:47.489 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 sudo[113169]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 sudo[113172]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-12T09:52:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 sudo[113172]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-12T09:52:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 sudo[113172]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-12T09:52:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 sudo[113172]: pam_unix(sudo:session): session closed for user root 2023-12-12T09:52:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3048406962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c3fe771-613f-4d10-956b-78db77d66068"}]: dispatch 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3048406962' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c3fe771-613f-4d10-956b-78db77d66068"}]': finished 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: osdmap e22: 4 total, 3 up, 4 in 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:52:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:52:47.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3048406962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c3fe771-613f-4d10-956b-78db77d66068"}]: dispatch 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3048406962' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c3fe771-613f-4d10-956b-78db77d66068"}]': finished 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: osdmap e22: 4 total, 3 up, 4 in 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:52:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:52:47.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:47.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-12T09:52:47.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-12T09:52:47.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:52:47.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3048406962' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c3fe771-613f-4d10-956b-78db77d66068"}]: dispatch 2023-12-12T09:52:47.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:52:47.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:52:47.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3048406962' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c3fe771-613f-4d10-956b-78db77d66068"}]': finished 2023-12-12T09:52:47.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: osdmap e22: 4 total, 3 up, 4 in 2023-12-12T09:52:47.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:52:47.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:52:47.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:52:47.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:52:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:48 smithi173 ceph-mon[103308]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:48 smithi173 ceph-mon[103308]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-12T09:52:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:48 smithi173 ceph-mon[103308]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-12T09:52:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:48 smithi173 ceph-mon[103308]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-12T09:52:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:52:48.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:52:48.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:52:48.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:48 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/230687293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:52:48.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:48 smithi173 ceph-mon[103308]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-12T09:52:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:48 smithi042 ceph-mon[98894]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:48 smithi042 ceph-mon[98894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-12T09:52:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:48 smithi042 ceph-mon[98894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-12T09:52:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:48 smithi042 ceph-mon[98894]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-12T09:52:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:52:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:52:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:52:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:48 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/230687293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:52:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:48 smithi042 ceph-mon[98894]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-12T09:52:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:48 smithi111 ceph-mon[102565]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:48 smithi111 ceph-mon[102565]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-12T09:52:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:48 smithi111 ceph-mon[102565]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-12T09:52:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:48 smithi111 ceph-mon[102565]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-12T09:52:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:52:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:52:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:52:48.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:48 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/230687293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:52:48.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:48 smithi111 ceph-mon[102565]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-12T09:52:49.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:49 smithi173 ceph-mon[103308]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:49.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:49 smithi042 ceph-mon[98894]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:49.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:49 smithi111 ceph-mon[102565]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:51.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:51 smithi173 ceph-mon[103308]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:51.853 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:51 smithi042 ceph-mon[98894]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:51 smithi111 ceph-mon[102565]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:52:53.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:53 smithi173 ceph-mon[103308]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 50 KiB/s wr, 4 op/s 2023-12-12T09:52:53.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:53 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-12T09:52:53.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:53 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:53.867 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:53 smithi042 ceph-mon[98894]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 50 KiB/s wr, 4 op/s 2023-12-12T09:52:53.867 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:53 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-12T09:52:53.868 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:53 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:53 smithi111 ceph-mon[102565]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.9 KiB/s rd, 50 KiB/s wr, 4 op/s 2023-12-12T09:52:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:53 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-12T09:52:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:53 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:52:54.701 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:54 smithi042 ceph-mon[98894]: Deploying daemon osd.3 on smithi042 2023-12-12T09:52:54.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:54 smithi173 ceph-mon[103308]: Deploying daemon osd.3 on smithi042 2023-12-12T09:52:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:54 smithi111 ceph-mon[102565]: Deploying daemon osd.3 on smithi042 2023-12-12T09:52:55.806 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:55 smithi042 ceph-mon[98894]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 36 KiB/s wr, 3 op/s 2023-12-12T09:52:55.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:55 smithi173 ceph-mon[103308]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 36 KiB/s wr, 3 op/s 2023-12-12T09:52:55.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:55 smithi111 ceph-mon[102565]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 36 KiB/s wr, 3 op/s 2023-12-12T09:52:57.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:57 smithi042 ceph-mon[98894]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2023-12-12T09:52:57.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:57 smithi173 ceph-mon[103308]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2023-12-12T09:52:57.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:57 smithi111 ceph-mon[102565]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 28 KiB/s wr, 2 op/s 2023-12-12T09:52:59.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:58 smithi042 ceph-mon[98894]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 909 B/s rd, 23 KiB/s wr, 2 op/s 2023-12-12T09:52:59.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:58 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:59.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:58 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:59.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:52:58 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:59.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:58 smithi111 ceph-mon[102565]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 909 B/s rd, 23 KiB/s wr, 2 op/s 2023-12-12T09:52:59.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:58 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:59.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:58 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:59.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:52:58 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:59.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:58 smithi173 ceph-mon[103308]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 909 B/s rd, 23 KiB/s wr, 2 op/s 2023-12-12T09:52:59.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:58 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:52:59.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:58 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:52:59.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:52:58 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:00.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.912+0000 7f68527fc700 1 -- 172.21.15.42:0/3459170000 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f685c061a10 con 0x7f68480a13f0 2023-12-12T09:53:00.915 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 3 on host 'smithi042' 2023-12-12T09:53:00.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f68480a13f0 msgr2=0x7f68480a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:00.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f68480a13f0 0x7f68480a38b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6858003820 tx=0x7f6858005040 comp rx=0 tx=0).stop 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c1031c0 msgr2=0x7f685c073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c1031c0 0x7f685c073f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f685400ada0 tx=0x7f685400c330 comp rx=0 tx=0).stop 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 shutdown_connections 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f68480a13f0 0x7f68480a38b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f685c1031c0 0x7f685c073f00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f685c101620 0x7f685c075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 --2- 172.21.15.42:0/3459170000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f685c102820 0x7f685c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.914+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 >> 172.21.15.42:0/3459170000 conn(0x7f685c0fcbe0 msgr2=0x7f685c105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.915+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 shutdown_connections 2023-12-12T09:53:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:00.915+0000 7f6863549700 1 -- 172.21.15.42:0/3459170000 wait complete. 2023-12-12T09:53:01.370 DEBUG:teuthology.orchestra.run.smithi042:osd.3> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.3.service 2023-12-12T09:53:01.372 INFO:tasks.cephadm:Deploying osd.4 on smithi111 with /dev/vg_nvme/lv_3... 2023-12-12T09:53:01.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-12T09:53:01.407 INFO:journalctl@ceph.osd.3.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:53:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-mon[98894]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-12T09:53:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:01 smithi111 ceph-mon[102565]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-12T09:53:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:01 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:01.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:01 smithi173 ceph-mon[103308]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-12T09:53:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:01 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:01.974 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T09:53:02.153 INFO:journalctl@ceph.osd.3.smithi042.stdout:Dec 12 09:53:01 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-3[126302]: 2023-12-12T09:53:01.825+0000 7f8d08252640 -1 osd.3 0 log_to_monitors true 2023-12-12T09:53:02.650 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:02 smithi042 ceph-mon[98894]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-12T09:53:02.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:02 smithi111 ceph-mon[102565]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-12T09:53:02.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:02 smithi173 ceph-mon[103308]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-12T09:53:03.417 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-12T09:53:03.417 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-12T09:53:03.417 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2023-12-12T09:53:03.417 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2023-12-12T09:53:03.417 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114345 s, 917 MB/s 2023-12-12T09:53:03.417 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2023-12-12T09:53:03.677 INFO:journalctl@ceph.osd.3.smithi042.stdout:Dec 12 09:53:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-3[126302]: 2023-12-12T09:53:03.422+0000 7f8cfd8c9700 -1 osd.3 0 waiting for initial osdmap 2023-12-12T09:53:03.677 INFO:journalctl@ceph.osd.3.smithi042.stdout:Dec 12 09:53:03 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-3[126302]: 2023-12-12T09:53:03.429+0000 7f8cf96c3700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:53:03.677 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:03 smithi042 ceph-mon[98894]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-12T09:53:03.678 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:03 smithi042 ceph-mon[98894]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-12T09:53:03.678 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:03 smithi042 ceph-mon[98894]: osdmap e23: 4 total, 3 up, 4 in 2023-12-12T09:53:03.678 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:03 smithi042 ceph-mon[98894]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:53:03.678 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:03 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:03.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:03 smithi111 ceph-mon[102565]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-12T09:53:03.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:03 smithi111 ceph-mon[102565]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-12T09:53:03.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:03 smithi111 ceph-mon[102565]: osdmap e23: 4 total, 3 up, 4 in 2023-12-12T09:53:03.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:03 smithi111 ceph-mon[102565]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:53:03.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:03 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:03.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:03 smithi173 ceph-mon[103308]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2023-12-12T09:53:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:03 smithi173 ceph-mon[103308]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-12T09:53:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:03 smithi173 ceph-mon[103308]: osdmap e23: 4 total, 3 up, 4 in 2023-12-12T09:53:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:03 smithi173 ceph-mon[103308]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:53:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:03 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:03.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi111:vg_nvme/lv_3 2023-12-12T09:53:04.431 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T09:53:04.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:53:04.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: osdmap e24: 4 total, 3 up, 4 in 2023-12-12T09:53:04.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:04.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:04.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:04.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:04.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:04.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:04.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:04 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: osdmap e24: 4 total, 3 up, 4 in 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:04 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:53:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: osdmap e24: 4 total, 3 up, 4 in 2023-12-12T09:53:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:04.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:04.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:04.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:04.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:04.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:04.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:04.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:04 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:05.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.498+0000 7fd4d7d45700 1 -- 172.21.15.111:0/4181784860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01016f0 msgr2=0x7fd4d0101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.498+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/4181784860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01016f0 0x7fd4d0101b50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00099c0 tx=0x7fd4c0009cd0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 -- 172.21.15.111:0/4181784860 shutdown_connections 2023-12-12T09:53:05.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/4181784860 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d0102090 0x7fd4d0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/4181784860 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d01004f0 0x7fd4d0100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/4181784860 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01016f0 0x7fd4d0101b50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 -- 172.21.15.111:0/4181784860 >> 172.21.15.111:0/4181784860 conn(0x7fd4d00fba90 msgr2=0x7fd4d00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 -- 172.21.15.111:0/4181784860 shutdown_connections 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 -- 172.21.15.111:0/4181784860 wait complete. 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 Processor -- start 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 -- start start 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01004f0 0x7fd4d006a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d01016f0 0x7fd4d019c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d0102090 0x7fd4d019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d019f020 con 0x7fd4d01016f0 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d019f190 con 0x7fd4d01004f0 2023-12-12T09:53:05.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d7d45700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d019f350 con 0x7fd4d0102090 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d5ae1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01004f0 0x7fd4d006a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d5ae1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01004f0 0x7fd4d006a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58110/0 (socket says 172.21.15.111:58110) 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.499+0000 7fd4d5ae1700 1 -- 172.21.15.111:0/384299446 learned_addr learned my addr 172.21.15.111:0/384299446 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d52e0700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d01016f0 0x7fd4d019c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d62e2700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d0102090 0x7fd4d019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d5ae1700 1 -- 172.21.15.111:0/384299446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d0102090 msgr2=0x7fd4d019e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d5ae1700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d0102090 0x7fd4d019e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d5ae1700 1 -- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d01016f0 msgr2=0x7fd4d019c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d5ae1700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d01016f0 0x7fd4d019c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d5ae1700 1 -- 172.21.15.111:0/384299446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c00095f0 con 0x7fd4d01004f0 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d5ae1700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01004f0 0x7fd4d006a810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00d980 tx=0x7fd4cc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4c6ffd700 1 -- 172.21.15.111:0/384299446 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc004020 con 0x7fd4d01004f0 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4c6ffd700 1 -- 172.21.15.111:0/384299446 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd4cc00f690 con 0x7fd4d01004f0 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d019f630 con 0x7fd4d01004f0 2023-12-12T09:53:05.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4c6ffd700 1 -- 172.21.15.111:0/384299446 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc003a60 con 0x7fd4d01004f0 2023-12-12T09:53:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.500+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d019fb80 con 0x7fd4d01004f0 2023-12-12T09:53:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.501+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d0107980 con 0x7fd4d01004f0 2023-12-12T09:53:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.502+0000 7fd4c6ffd700 1 -- 172.21.15.111:0/384299446 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd4cc00fb00 con 0x7fd4d01004f0 2023-12-12T09:53:05.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.503+0000 7fd4c6ffd700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4bc0a1500 0x7fd4bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:05.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.503+0000 7fd4c6ffd700 1 -- 172.21.15.111:0/384299446 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3698+0+0 (secure 0 0 0) 0x7fd4cc0cb630 con 0x7fd4d01004f0 2023-12-12T09:53:05.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.503+0000 7fd4d52e0700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4bc0a1500 0x7fd4bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:05.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.503+0000 7fd4d52e0700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4bc0a1500 0x7fd4bc0a39c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009990 tx=0x7fd4c0005bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:05.508 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.507+0000 7fd4c6ffd700 1 -- 172.21.15.111:0/384299446 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd4cc095e90 con 0x7fd4d01004f0 2023-12-12T09:53:05.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:05.705+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd4d0061a10 con 0x7fd4bc0a1500 2023-12-12T09:53:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:05 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:53:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:05 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:53:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:05 smithi111 ceph-mon[102565]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:53:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:05 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi042 2023-12-12T09:53:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:05 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi042 to 8586M 2023-12-12T09:53:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:05 smithi111 ceph-mon[102565]: osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] boot 2023-12-12T09:53:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:05 smithi111 ceph-mon[102565]: osdmap e25: 4 total, 4 up, 4 in 2023-12-12T09:53:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:05.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:05 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:53:05.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:05 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:53:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:05 smithi173 ceph-mon[103308]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:53:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:05 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi042 2023-12-12T09:53:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:05 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi042 to 8586M 2023-12-12T09:53:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:05 smithi173 ceph-mon[103308]: osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] boot 2023-12-12T09:53:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:05 smithi173 ceph-mon[103308]: osdmap e25: 4 total, 4 up, 4 in 2023-12-12T09:53:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:05.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:05 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:53:05.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:05 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:53:05.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:05 smithi042 ceph-mon[98894]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2023-12-12T09:53:05.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:05 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi042 2023-12-12T09:53:05.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:05 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi042 to 8586M 2023-12-12T09:53:05.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:05.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:05 smithi042 ceph-mon[98894]: osd.3 [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] boot 2023-12-12T09:53:05.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:05 smithi042 ceph-mon[98894]: osdmap e25: 4 total, 4 up, 4 in 2023-12-12T09:53:05.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:53:06.725 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:06 smithi111 ceph-mon[102565]: osdmap e26: 4 total, 4 up, 4 in 2023-12-12T09:53:06.725 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:53:06.725 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:53:06.725 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:06.725 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:06 smithi111 ceph-mon[102565]: osdmap e27: 4 total, 4 up, 4 in 2023-12-12T09:53:06.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:06 smithi173 ceph-mon[103308]: osdmap e26: 4 total, 4 up, 4 in 2023-12-12T09:53:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:53:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:53:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:06 smithi173 ceph-mon[103308]: osdmap e27: 4 total, 4 up, 4 in 2023-12-12T09:53:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:06 smithi042 ceph-mon[98894]: osdmap e26: 4 total, 4 up, 4 in 2023-12-12T09:53:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:53:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:53:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:06 smithi042 ceph-mon[98894]: osdmap e27: 4 total, 4 up, 4 in 2023-12-12T09:53:07.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:07 smithi111 ceph-mon[102565]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:53:07.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:07 smithi111 ceph-mon[102565]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:07.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:07 smithi173 ceph-mon[103308]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:53:07.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:07 smithi173 ceph-mon[103308]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:07.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:07 smithi042 ceph-mon[98894]: from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:53:07.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:07 smithi042 ceph-mon[98894]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:08.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:08 smithi111 ceph-mon[102565]: osdmap e28: 4 total, 4 up, 4 in 2023-12-12T09:53:08.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:08 smithi111 ceph-mon[102565]: from='client.? 172.21.15.111:0/1086774533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "664d9eb1-f016-4943-a329-e18c4a7d9377"}]: dispatch 2023-12-12T09:53:08.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:08 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "664d9eb1-f016-4943-a329-e18c4a7d9377"}]: dispatch 2023-12-12T09:53:08.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:08 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "664d9eb1-f016-4943-a329-e18c4a7d9377"}]': finished 2023-12-12T09:53:08.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:08 smithi111 ceph-mon[102565]: osdmap e29: 5 total, 4 up, 5 in 2023-12-12T09:53:08.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:08 smithi173 ceph-mon[103308]: osdmap e28: 4 total, 4 up, 4 in 2023-12-12T09:53:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:08 smithi173 ceph-mon[103308]: from='client.? 172.21.15.111:0/1086774533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "664d9eb1-f016-4943-a329-e18c4a7d9377"}]: dispatch 2023-12-12T09:53:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:08 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "664d9eb1-f016-4943-a329-e18c4a7d9377"}]: dispatch 2023-12-12T09:53:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:08 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "664d9eb1-f016-4943-a329-e18c4a7d9377"}]': finished 2023-12-12T09:53:08.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:08 smithi173 ceph-mon[103308]: osdmap e29: 5 total, 4 up, 5 in 2023-12-12T09:53:08.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:08 smithi042 ceph-mon[98894]: osdmap e28: 4 total, 4 up, 4 in 2023-12-12T09:53:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:08 smithi042 ceph-mon[98894]: from='client.? 172.21.15.111:0/1086774533' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "664d9eb1-f016-4943-a329-e18c4a7d9377"}]: dispatch 2023-12-12T09:53:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:08 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "664d9eb1-f016-4943-a329-e18c4a7d9377"}]: dispatch 2023-12-12T09:53:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:08 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "664d9eb1-f016-4943-a329-e18c4a7d9377"}]': finished 2023-12-12T09:53:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:08 smithi042 ceph-mon[98894]: osdmap e29: 5 total, 4 up, 5 in 2023-12-12T09:53:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:09.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:09 smithi111 ceph-mon[102565]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:09.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:09 smithi111 ceph-mon[102565]: from='client.? 172.21.15.111:0/771213246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:53:09.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:09 smithi173 ceph-mon[103308]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:09.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:09 smithi173 ceph-mon[103308]: from='client.? 172.21.15.111:0/771213246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:53:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:09 smithi042 ceph-mon[98894]: pgmap v98: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:09 smithi042 ceph-mon[98894]: from='client.? 172.21.15.111:0/771213246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:53:11.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:11 smithi111 ceph-mon[102565]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:11.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:11 smithi173 ceph-mon[103308]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:11.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:11 smithi042 ceph-mon[98894]: pgmap v100: 1 pgs: 1 remapped+peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:13.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:13 smithi111 ceph-mon[102565]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-12T09:53:13.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:13 smithi173 ceph-mon[103308]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-12T09:53:13.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:13 smithi042 ceph-mon[98894]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-12T09:53:15.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:15 smithi111 ceph-mon[102565]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-12-12T09:53:15.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-12T09:53:15.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:15.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:15 smithi111 ceph-mon[102565]: Deploying daemon osd.4 on smithi111 2023-12-12T09:53:15.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:15 smithi173 ceph-mon[103308]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-12-12T09:53:15.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-12T09:53:15.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:15.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:15 smithi173 ceph-mon[103308]: Deploying daemon osd.4 on smithi111 2023-12-12T09:53:15.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:15 smithi042 ceph-mon[98894]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2023-12-12T09:53:15.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-12T09:53:15.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:15.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:15 smithi042 ceph-mon[98894]: Deploying daemon osd.4 on smithi111 2023-12-12T09:53:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:17 smithi111 ceph-mon[102565]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-12-12T09:53:17.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:17 smithi173 ceph-mon[103308]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-12-12T09:53:17.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:17 smithi042 ceph-mon[98894]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-12-12T09:53:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:19 smithi111 ceph-mon[102565]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-12T09:53:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:19.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:19 smithi173 ceph-mon[103308]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-12T09:53:19.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:19.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:19 smithi042 ceph-mon[98894]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-12T09:53:19.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:19.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:19.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:21 smithi111 ceph-mon[102565]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-12T09:53:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:21.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:21.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:21.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:21 smithi173 ceph-mon[103308]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-12T09:53:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:21.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.886+0000 7fd4c6ffd700 1 -- 172.21.15.111:0/384299446 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd4d0061a10 con 0x7fd4bc0a1500 2023-12-12T09:53:21.888 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 4 on host 'smithi111' 2023-12-12T09:53:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4bc0a1500 msgr2=0x7fd4bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4bc0a1500 0x7fd4bc0a39c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009990 tx=0x7fd4c0005bc0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01004f0 msgr2=0x7fd4d006a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01004f0 0x7fd4d006a810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00d980 tx=0x7fd4cc00dc90 comp rx=0 tx=0).stop 2023-12-12T09:53:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 shutdown_connections 2023-12-12T09:53:21.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4bc0a1500 0x7fd4bc0a39c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4d01004f0 0x7fd4d006a810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d0102090 0x7fd4d019e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 --2- 172.21.15.111:0/384299446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4d01016f0 0x7fd4d019c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:21.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 >> 172.21.15.111:0/384299446 conn(0x7fd4d00fba90 msgr2=0x7fd4d0104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:21.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 shutdown_connections 2023-12-12T09:53:21.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:53:21.888+0000 7fd4d7d45700 1 -- 172.21.15.111:0/384299446 wait complete. 2023-12-12T09:53:21.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:21 smithi042 ceph-mon[98894]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-12T09:53:21.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:21.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:21.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:21.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:21.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:22.323 DEBUG:teuthology.orchestra.run.smithi111:osd.4> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.4.service 2023-12-12T09:53:22.325 INFO:tasks.cephadm:Deploying osd.5 on smithi173 with /dev/vg_nvme/lv_3... 2023-12-12T09:53:22.325 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-12T09:53:22.376 INFO:journalctl@ceph.osd.4.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:53:22.937 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:53:23.137 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 09:53:23 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4[116403]: 2023-12-12T09:53:23.038+0000 7f7297dc7640 -1 osd.4 0 log_to_monitors true 2023-12-12T09:53:23.137 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:22 smithi111 ceph-mon[102565]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-12T09:53:23.137 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:22 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:23.137 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:22 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:23.138 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:22 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:23.138 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:22 smithi173 ceph-mon[103308]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-12T09:53:23.138 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:22 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:23.138 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:22 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:23.138 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:22 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:23.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:22 smithi042 ceph-mon[98894]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-12T09:53:23.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:22 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:23.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:22 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:23.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:22 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:23.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:23 smithi111 ceph-mon[102565]: from='osd.4 [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-12T09:53:23.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:23 smithi111 ceph-mon[102565]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-12T09:53:24.152 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:23 smithi042 ceph-mon[98894]: from='osd.4 [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-12T09:53:24.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:23 smithi042 ceph-mon[98894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-12T09:53:24.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:23 smithi173 ceph-mon[103308]: from='osd.4 [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-12T09:53:24.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:23 smithi173 ceph-mon[103308]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-12T09:53:24.435 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-12T09:53:24.435 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-12T09:53:24.435 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2023-12-12T09:53:24.435 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2023-12-12T09:53:24.435 INFO:teuthology.orchestra.run.smithi173.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0109873 s, 954 MB/s 2023-12-12T09:53:24.435 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2023-12-12T09:53:24.894 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi173:vg_nvme/lv_3 2023-12-12T09:53:25.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:24 smithi042 ceph-mon[98894]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:25.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:24 smithi042 ceph-mon[98894]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-12T09:53:25.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:24 smithi042 ceph-mon[98894]: osdmap e30: 5 total, 4 up, 5 in 2023-12-12T09:53:25.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:24 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:25.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:24 smithi042 ceph-mon[98894]: from='osd.4 [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:53:25.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:24 smithi042 ceph-mon[98894]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:53:25.247 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 09:53:24 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4[116403]: 2023-12-12T09:53:24.865+0000 7f728d43e700 -1 osd.4 0 waiting for initial osdmap 2023-12-12T09:53:25.247 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 09:53:24 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4[116403]: 2023-12-12T09:53:24.872+0000 7f7289238700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:53:25.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:24 smithi111 ceph-mon[102565]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:25.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:24 smithi111 ceph-mon[102565]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-12T09:53:25.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:24 smithi111 ceph-mon[102565]: osdmap e30: 5 total, 4 up, 5 in 2023-12-12T09:53:25.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:24 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:25.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:24 smithi111 ceph-mon[102565]: from='osd.4 [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:53:25.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:24 smithi111 ceph-mon[102565]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:53:25.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:24 smithi173 ceph-mon[103308]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:24 smithi173 ceph-mon[103308]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-12T09:53:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:24 smithi173 ceph-mon[103308]: osdmap e30: 5 total, 4 up, 5 in 2023-12-12T09:53:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:24 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:24 smithi173 ceph-mon[103308]: from='osd.4 [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:53:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:24 smithi173 ceph-mon[103308]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:53:25.489 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:53:26.023 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: osdmap e31: 5 total, 4 up, 5 in 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi111 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi111 to 8584M 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.024 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:26.025 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:26.025 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:53:26.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: osdmap e31: 5 total, 4 up, 5 in 2023-12-12T09:53:26.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:26.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:26.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi111 2023-12-12T09:53:26.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:26.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:26.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi111 to 8584M 2023-12-12T09:53:26.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:26.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:26.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:53:26.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: osdmap e31: 5 total, 4 up, 5 in 2023-12-12T09:53:26.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:26.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:26.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi111 2023-12-12T09:53:26.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:26.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:26.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi111 to 8584M 2023-12-12T09:53:26.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:26.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:26.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:26.761 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 -- 172.21.15.173:0/2567364224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18100510 msgr2=0x7fcf18100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.761 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 --2- 172.21.15.173:0/2567364224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18100510 0x7fcf18100930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009990 tx=0x7fcf0c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.761 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 -- 172.21.15.173:0/2567364224 shutdown_connections 2023-12-12T09:53:26.761 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 --2- 172.21.15.173:0/2567364224 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf181020b0 0x7fcf18108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.761 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 --2- 172.21.15.173:0/2567364224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18100510 0x7fcf18100930 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.761 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 --2- 172.21.15.173:0/2567364224 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf18101710 0x7fcf18101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.761 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 -- 172.21.15.173:0/2567364224 >> 172.21.15.173:0/2567364224 conn(0x7fcf180fbad0 msgr2=0x7fcf180fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:26.762 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 -- 172.21.15.173:0/2567364224 shutdown_connections 2023-12-12T09:53:26.762 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 -- 172.21.15.173:0/2567364224 wait complete. 2023-12-12T09:53:26.762 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 Processor -- start 2023-12-12T09:53:26.762 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.759+0000 7fcf1e365700 1 -- start start 2023-12-12T09:53:26.763 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf1e365700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf18100510 0x7fcf18104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf1e365700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18101710 0x7fcf18102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf1e365700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf181020b0 0x7fcf18103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf1e365700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181037d0 con 0x7fcf18100510 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf1e365700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf18103940 con 0x7fcf181020b0 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf1e365700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf181a15b0 con 0x7fcf18101710 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf177fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18101710 0x7fcf18102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf177fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18101710 0x7fcf18102af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42594/0 (socket says 172.21.15.173:42594) 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf177fe700 1 -- 172.21.15.173:0/3007992152 learned_addr learned my addr 172.21.15.173:0/3007992152 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf17fff700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf18100510 0x7fcf18104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf1c902700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf181020b0 0x7fcf18103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf177fe700 1 -- 172.21.15.173:0/3007992152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf181020b0 msgr2=0x7fcf18103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf177fe700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf181020b0 0x7fcf18103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf177fe700 1 -- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf18100510 msgr2=0x7fcf18104450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:26.764 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf177fe700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf18100510 0x7fcf18104450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:26.765 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf177fe700 1 -- 172.21.15.173:0/3007992152 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0c0095f0 con 0x7fcf18101710 2023-12-12T09:53:26.765 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf177fe700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18101710 0x7fcf18102af0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800d950 tx=0x7fcf0800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:26.765 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.760+0000 7fcf157fa700 1 -- 172.21.15.173:0/3007992152 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800f890 con 0x7fcf18101710 2023-12-12T09:53:26.765 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.761+0000 7fcf157fa700 1 -- 172.21.15.173:0/3007992152 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf08010460 con 0x7fcf18101710 2023-12-12T09:53:26.765 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.761+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf181a1840 con 0x7fcf18101710 2023-12-12T09:53:26.765 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.761+0000 7fcf157fa700 1 -- 172.21.15.173:0/3007992152 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0800e610 con 0x7fcf18101710 2023-12-12T09:53:26.765 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.761+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf181a1d40 con 0x7fcf18101710 2023-12-12T09:53:26.765 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.761+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1804f290 con 0x7fcf18101710 2023-12-12T09:53:26.766 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.762+0000 7fcf157fa700 1 -- 172.21.15.173:0/3007992152 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fcf080108d0 con 0x7fcf18101710 2023-12-12T09:53:26.766 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.763+0000 7fcf157fa700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf000a1480 0x7fcf000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:26.766 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.763+0000 7fcf157fa700 1 -- 172.21.15.173:0/3007992152 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4101+0+0 (secure 0 0 0) 0x7fcf080cba20 con 0x7fcf18101710 2023-12-12T09:53:26.766 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.763+0000 7fcf17fff700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf000a1480 0x7fcf000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:26.766 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.764+0000 7fcf17fff700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf000a1480 0x7fcf000a3940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c006010 tx=0x7fcf0c003830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:26.769 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.768+0000 7fcf157fa700 1 -- 172.21.15.173:0/3007992152 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcf080960f0 con 0x7fcf18101710 2023-12-12T09:53:26.968 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:26.966+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf18069680 con 0x7fcf000a1480 2023-12-12T09:53:27.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:26 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:53:27.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:26 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:53:27.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:26 smithi042 ceph-mon[98894]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:27.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:26 smithi042 ceph-mon[98894]: osd.4 [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] boot 2023-12-12T09:53:27.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:26 smithi042 ceph-mon[98894]: osdmap e32: 5 total, 5 up, 5 in 2023-12-12T09:53:27.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:26 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:26 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:53:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:26 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:53:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:26 smithi111 ceph-mon[102565]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:26 smithi111 ceph-mon[102565]: osd.4 [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] boot 2023-12-12T09:53:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:26 smithi111 ceph-mon[102565]: osdmap e32: 5 total, 5 up, 5 in 2023-12-12T09:53:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:27.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:26 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:53:27.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:26 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:53:27.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:26 smithi173 ceph-mon[103308]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2023-12-12T09:53:27.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:26 smithi173 ceph-mon[103308]: osd.4 [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] boot 2023-12-12T09:53:27.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:26 smithi173 ceph-mon[103308]: osdmap e32: 5 total, 5 up, 5 in 2023-12-12T09:53:27.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:26 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:53:28.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:27 smithi173 ceph-mon[103308]: osdmap e33: 5 total, 5 up, 5 in 2023-12-12T09:53:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:27 smithi173 ceph-mon[103308]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:53:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:53:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:53:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:27 smithi111 ceph-mon[102565]: osdmap e33: 5 total, 5 up, 5 in 2023-12-12T09:53:28.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:27 smithi111 ceph-mon[102565]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:53:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:53:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:53:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:28.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:27 smithi042 ceph-mon[98894]: osdmap e33: 5 total, 5 up, 5 in 2023-12-12T09:53:28.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:27 smithi042 ceph-mon[98894]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:53:28.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:53:28.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:53:28.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:29.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:29 smithi111 ceph-mon[102565]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:29.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:29 smithi173 ceph-mon[103308]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:29 smithi042 ceph-mon[98894]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:30 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/510388022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13"}]: dispatch 2023-12-12T09:53:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:30 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13"}]: dispatch 2023-12-12T09:53:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:30 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13"}]': finished 2023-12-12T09:53:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:30 smithi111 ceph-mon[102565]: osdmap e34: 6 total, 5 up, 6 in 2023-12-12T09:53:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:30.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:30 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/4248685175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:53:30.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:30 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/510388022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13"}]: dispatch 2023-12-12T09:53:30.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:30 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13"}]: dispatch 2023-12-12T09:53:30.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:30 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13"}]': finished 2023-12-12T09:53:30.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:30 smithi173 ceph-mon[103308]: osdmap e34: 6 total, 5 up, 6 in 2023-12-12T09:53:30.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:30.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:30 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/4248685175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:53:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:30 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/510388022' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13"}]: dispatch 2023-12-12T09:53:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:30 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13"}]: dispatch 2023-12-12T09:53:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:30 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13"}]': finished 2023-12-12T09:53:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:30 smithi042 ceph-mon[98894]: osdmap e34: 6 total, 5 up, 6 in 2023-12-12T09:53:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:30 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/4248685175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:53:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:31 smithi111 ceph-mon[102565]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:31.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:31 smithi173 ceph-mon[103308]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:31.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:31 smithi042 ceph-mon[98894]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:33.548 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:33 smithi173 ceph-mon[103308]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:33.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:33 smithi111 ceph-mon[102565]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:33.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:33 smithi042 ceph-mon[98894]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:35.697 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:35 smithi173 ceph-mon[103308]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:35 smithi111 ceph-mon[102565]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:35.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:35 smithi042 ceph-mon[98894]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:36.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:36 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-12T09:53:36.888 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:36 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:36.889 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:36 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-12T09:53:36.889 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:36 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:36.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:36 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-12T09:53:36.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:36 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:37 smithi111 ceph-mon[102565]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:37 smithi111 ceph-mon[102565]: Deploying daemon osd.5 on smithi173 2023-12-12T09:53:37.819 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:37 smithi173 ceph-mon[103308]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:37.819 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:37 smithi173 ceph-mon[103308]: Deploying daemon osd.5 on smithi173 2023-12-12T09:53:37.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:37 smithi042 ceph-mon[98894]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:37.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:37 smithi042 ceph-mon[98894]: Deploying daemon osd.5 on smithi173 2023-12-12T09:53:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:39 smithi111 ceph-mon[102565]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:39 smithi173 ceph-mon[103308]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:39.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:39 smithi042 ceph-mon[98894]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:41 smithi111 ceph-mon[102565]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:41 smithi173 ceph-mon[103308]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:41 smithi042 ceph-mon[98894]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:43.388 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.385+0000 7fcf157fa700 1 -- 172.21.15.173:0/3007992152 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcf18069680 con 0x7fcf000a1480 2023-12-12T09:53:43.388 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 5 on host 'smithi173' 2023-12-12T09:53:43.390 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf000a1480 msgr2=0x7fcf000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.390 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf000a1480 0x7fcf000a3940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c006010 tx=0x7fcf0c003830 comp rx=0 tx=0).stop 2023-12-12T09:53:43.390 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18101710 msgr2=0x7fcf18102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:43.390 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18101710 0x7fcf18102af0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcf0800d950 tx=0x7fcf0800dc60 comp rx=0 tx=0).stop 2023-12-12T09:53:43.390 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 shutdown_connections 2023-12-12T09:53:43.390 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf000a1480 0x7fcf000a3940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.390 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf181020b0 0x7fcf18103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.390 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf18100510 0x7fcf18104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.391 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 --2- 172.21.15.173:0/3007992152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf18101710 0x7fcf18102af0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:43.391 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.387+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 >> 172.21.15.173:0/3007992152 conn(0x7fcf180fbad0 msgr2=0x7fcf18068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:43.391 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.388+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 shutdown_connections 2023-12-12T09:53:43.391 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:53:43.388+0000 7fcf1e365700 1 -- 172.21.15.173:0/3007992152 wait complete. 2023-12-12T09:53:43.548 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:43 smithi173 ceph-mon[103308]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:43.548 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.548 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.548 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:43.548 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:43.549 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.549 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.549 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.549 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:43 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:43 smithi111 ceph-mon[102565]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:43 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:43.882 DEBUG:teuthology.orchestra.run.smithi173:osd.5> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.5.service 2023-12-12T09:53:43.887 INFO:tasks.cephadm:Deploying osd.6 on smithi042 with /dev/vg_nvme/lv_2... 2023-12-12T09:53:43.888 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-12T09:53:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:43 smithi042 ceph-mon[98894]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:43 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:53:43.917 INFO:journalctl@ceph.osd.5.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:53:44.530 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:53:44.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:44 smithi173 ceph-mon[103308]: from='osd.5 [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-12T09:53:44.645 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:44 smithi173 ceph-mon[103308]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-12T09:53:44.645 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 09:53:44 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T09:53:44.372+0000 7f4fc729d640 -1 osd.5 0 log_to_monitors true 2023-12-12T09:53:44.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:44 smithi111 ceph-mon[102565]: from='osd.5 [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-12T09:53:44.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:44 smithi111 ceph-mon[102565]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-12T09:53:44.753 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:44 smithi042 ceph-mon[98894]: from='osd.5 [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-12T09:53:44.753 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:44 smithi042 ceph-mon[98894]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-12T09:53:45.621 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:45 smithi042 ceph-mon[98894]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:45.621 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:45 smithi042 ceph-mon[98894]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-12T09:53:45.621 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:45 smithi042 ceph-mon[98894]: osdmap e35: 6 total, 5 up, 6 in 2023-12-12T09:53:45.621 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:45 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:45.621 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:45 smithi042 ceph-mon[98894]: from='osd.5 [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:53:45.622 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:45 smithi042 ceph-mon[98894]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:53:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:45 smithi111 ceph-mon[102565]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:45 smithi111 ceph-mon[102565]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-12T09:53:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:45 smithi111 ceph-mon[102565]: osdmap e35: 6 total, 5 up, 6 in 2023-12-12T09:53:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:45 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:45 smithi111 ceph-mon[102565]: from='osd.5 [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:53:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:45 smithi111 ceph-mon[102565]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:53:45.822 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 09:53:45 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T09:53:45.484+0000 7f4fbc914700 -1 osd.5 0 waiting for initial osdmap 2023-12-12T09:53:45.823 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 09:53:45 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T09:53:45.490+0000 7f4fb5f09700 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:53:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:45 smithi173 ceph-mon[103308]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:45 smithi173 ceph-mon[103308]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-12T09:53:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:45 smithi173 ceph-mon[103308]: osdmap e35: 6 total, 5 up, 6 in 2023-12-12T09:53:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:45 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:45 smithi173 ceph-mon[103308]: from='osd.5 [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:53:45.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:45 smithi173 ceph-mon[103308]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:53:46.205 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-12T09:53:46.205 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-12T09:53:46.205 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2023-12-12T09:53:46.205 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2023-12-12T09:53:46.205 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135597 s, 773 MB/s 2023-12-12T09:53:46.205 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: osdmap e36: 6 total, 5 up, 6 in 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:46.584 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:46 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.585 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_2 2023-12-12T09:53:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:53:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: osdmap e36: 6 total, 5 up, 6 in 2023-12-12T09:53:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:46.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:46.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:46 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: osdmap e36: 6 total, 5 up, 6 in 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:53:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:46 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:53:47.189 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:53:47.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:47 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:53:47.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:47 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:53:47.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:47 smithi042 ceph-mon[98894]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:47.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:47 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi173 2023-12-12T09:53:47.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:47 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi173 to 8584M 2023-12-12T09:53:47.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:47.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:47 smithi042 ceph-mon[98894]: osd.5 [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] boot 2023-12-12T09:53:47.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:47 smithi042 ceph-mon[98894]: osdmap e37: 6 total, 6 up, 6 in 2023-12-12T09:53:47.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:47 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:53:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:47 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:53:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:47 smithi111 ceph-mon[102565]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:47 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi173 2023-12-12T09:53:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:47 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi173 to 8584M 2023-12-12T09:53:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:47 smithi111 ceph-mon[102565]: osd.5 [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] boot 2023-12-12T09:53:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:47 smithi111 ceph-mon[102565]: osdmap e37: 6 total, 6 up, 6 in 2023-12-12T09:53:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:47.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:47 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:53:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:47 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:53:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:47 smithi173 ceph-mon[103308]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2023-12-12T09:53:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:47 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi173 2023-12-12T09:53:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:47 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi173 to 8584M 2023-12-12T09:53:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:47 smithi173 ceph-mon[103308]: osd.5 [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] boot 2023-12-12T09:53:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:47 smithi173 ceph-mon[103308]: osdmap e37: 6 total, 6 up, 6 in 2023-12-12T09:53:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:53:48.406 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.402+0000 7f88d2aae700 1 -- 172.21.15.42:0/31550246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc101710 msgr2=0x7f88cc101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.406 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.402+0000 7f88d2aae700 1 --2- 172.21.15.42:0/31550246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc101710 0x7f88cc101b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f88b80099c0 tx=0x7f88b8009cd0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.406 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.402+0000 7f88d2aae700 1 -- 172.21.15.42:0/31550246 shutdown_connections 2023-12-12T09:53:48.406 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.402+0000 7f88d2aae700 1 --2- 172.21.15.42:0/31550246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88cc1020b0 0x7f88cc108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.402+0000 7f88d2aae700 1 --2- 172.21.15.42:0/31550246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88cc100510 0x7f88cc100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.402+0000 7f88d2aae700 1 --2- 172.21.15.42:0/31550246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc101710 0x7f88cc101b70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.402+0000 7f88d2aae700 1 -- 172.21.15.42:0/31550246 >> 172.21.15.42:0/31550246 conn(0x7f88cc0fbad0 msgr2=0x7f88cc0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.402+0000 7f88d2aae700 1 -- 172.21.15.42:0/31550246 shutdown_connections 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.402+0000 7f88d2aae700 1 -- 172.21.15.42:0/31550246 wait complete. 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d2aae700 1 Processor -- start 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d2aae700 1 -- start start 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d2aae700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88cc100510 0x7f88cc199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d2aae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88cc101710 0x7f88cc199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d2aae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc1020b0 0x7f88cc1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d2aae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1a1240 con 0x7f88cc1020b0 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d2aae700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1a1380 con 0x7f88cc100510 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d2aae700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1a1510 con 0x7f88cc101710 2023-12-12T09:53:48.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d104b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc1020b0 0x7f88cc1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d104b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc1020b0 0x7f88cc1a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60950/0 (socket says 172.21.15.42:60950) 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d104b700 1 -- 172.21.15.42:0/2133985985 learned_addr learned my addr 172.21.15.42:0/2133985985 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88cbfff700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88cc101710 0x7f88cc199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d084a700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88cc100510 0x7f88cc199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d104b700 1 -- 172.21.15.42:0/2133985985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88cc101710 msgr2=0x7f88cc199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d104b700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88cc101710 0x7f88cc199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d104b700 1 -- 172.21.15.42:0/2133985985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88cc100510 msgr2=0x7f88cc199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d104b700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88cc100510 0x7f88cc199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.403+0000 7f88d104b700 1 -- 172.21.15.42:0/2133985985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80095f0 con 0x7f88cc1020b0 2023-12-12T09:53:48.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.404+0000 7f88d084a700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88cc100510 0x7f88cc199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:53:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.404+0000 7f88d104b700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc1020b0 0x7f88cc1a0c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88c4007f10 tx=0x7f88c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.404+0000 7f88c9ffb700 1 -- 172.21.15.42:0/2133985985 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4017070 con 0x7f88cc1020b0 2023-12-12T09:53:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.404+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc1a17f0 con 0x7f88cc1020b0 2023-12-12T09:53:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.404+0000 7f88c9ffb700 1 -- 172.21.15.42:0/2133985985 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f88c400f040 con 0x7f88cc1020b0 2023-12-12T09:53:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.404+0000 7f88c9ffb700 1 -- 172.21.15.42:0/2133985985 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4005740 con 0x7f88cc1020b0 2023-12-12T09:53:48.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.404+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc1a1cf0 con 0x7f88cc1020b0 2023-12-12T09:53:48.410 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.405+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc04f290 con 0x7f88cc1020b0 2023-12-12T09:53:48.410 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.406+0000 7f88c9ffb700 1 -- 172.21.15.42:0/2133985985 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f88c40058a0 con 0x7f88cc1020b0 2023-12-12T09:53:48.410 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.407+0000 7f88c9ffb700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f88bc0a1500 0x7f88bc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:53:48.410 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.407+0000 7f88d084a700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f88bc0a1500 0x7f88bc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:53:48.410 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.407+0000 7f88c9ffb700 1 -- 172.21.15.42:0/2133985985 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f88c40cbd30 con 0x7f88cc1020b0 2023-12-12T09:53:48.410 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.407+0000 7f88d084a700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f88bc0a1500 0x7f88bc0a39c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f88cc101570 tx=0x7f88c0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:53:48.412 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.410+0000 7f88c9ffb700 1 -- 172.21.15.42:0/2133985985 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f88c4096190 con 0x7f88cc1020b0 2023-12-12T09:53:48.625 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:53:48.623+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f88cc069f10 con 0x7f88bc0a1500 2023-12-12T09:53:48.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:48 smithi173 ceph-mon[103308]: osdmap e38: 6 total, 6 up, 6 in 2023-12-12T09:53:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:48 smithi042 ceph-mon[98894]: osdmap e38: 6 total, 6 up, 6 in 2023-12-12T09:53:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:48 smithi111 ceph-mon[102565]: osdmap e38: 6 total, 6 up, 6 in 2023-12-12T09:53:49.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:49 smithi042 ceph-mon[98894]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:53:49.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:49 smithi042 ceph-mon[98894]: osdmap e39: 6 total, 6 up, 6 in 2023-12-12T09:53:49.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:49 smithi042 ceph-mon[98894]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:53:49.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:49 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:53:49.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:49 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:53:49.779 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:49 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:49.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:49 smithi173 ceph-mon[103308]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:53:49.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:49 smithi173 ceph-mon[103308]: osdmap e39: 6 total, 6 up, 6 in 2023-12-12T09:53:49.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:49 smithi173 ceph-mon[103308]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:53:49.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:49 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:53:49.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:49 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:53:49.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:49 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:50.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:49 smithi111 ceph-mon[102565]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:53:50.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:49 smithi111 ceph-mon[102565]: osdmap e39: 6 total, 6 up, 6 in 2023-12-12T09:53:50.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:49 smithi111 ceph-mon[102565]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:53:50.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:49 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:53:50.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:49 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:53:50.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:49 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:50.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:50 smithi173 ceph-mon[103308]: osdmap e40: 6 total, 6 up, 6 in 2023-12-12T09:53:50.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:50 smithi042 ceph-mon[98894]: osdmap e40: 6 total, 6 up, 6 in 2023-12-12T09:53:50.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:50 smithi111 ceph-mon[102565]: osdmap e40: 6 total, 6 up, 6 in 2023-12-12T09:53:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:51 smithi173 ceph-mon[103308]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:53:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:51 smithi173 ceph-mon[103308]: osdmap e41: 6 total, 6 up, 6 in 2023-12-12T09:53:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:51 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1696916761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1cf8ac5-f733-425d-9148-02d06c5871e5"}]: dispatch 2023-12-12T09:53:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:51 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1696916761' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1cf8ac5-f733-425d-9148-02d06c5871e5"}]': finished 2023-12-12T09:53:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:51 smithi173 ceph-mon[103308]: osdmap e42: 7 total, 6 up, 7 in 2023-12-12T09:53:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:53:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:51 smithi042 ceph-mon[98894]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:53:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:51 smithi042 ceph-mon[98894]: osdmap e41: 6 total, 6 up, 6 in 2023-12-12T09:53:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:51 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1696916761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1cf8ac5-f733-425d-9148-02d06c5871e5"}]: dispatch 2023-12-12T09:53:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:51 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1696916761' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1cf8ac5-f733-425d-9148-02d06c5871e5"}]': finished 2023-12-12T09:53:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:51 smithi042 ceph-mon[98894]: osdmap e42: 7 total, 6 up, 7 in 2023-12-12T09:53:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:53:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:51 smithi111 ceph-mon[102565]: pgmap v131: 1 pgs: 1 peering; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:53:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:51 smithi111 ceph-mon[102565]: osdmap e41: 6 total, 6 up, 6 in 2023-12-12T09:53:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:51 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1696916761' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1cf8ac5-f733-425d-9148-02d06c5871e5"}]: dispatch 2023-12-12T09:53:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:51 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1696916761' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1cf8ac5-f733-425d-9148-02d06c5871e5"}]': finished 2023-12-12T09:53:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:51 smithi111 ceph-mon[102565]: osdmap e42: 7 total, 6 up, 7 in 2023-12-12T09:53:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:53:52.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:52 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3282656490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:53:52.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:52 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3282656490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:53:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:52 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3282656490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:53:53.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:53 smithi173 ceph-mon[103308]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-12T09:53:53.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:53 smithi042 ceph-mon[98894]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-12T09:53:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:53 smithi111 ceph-mon[102565]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-12T09:53:55.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:55 smithi173 ceph-mon[103308]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-12-12T09:53:55.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:55 smithi042 ceph-mon[98894]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-12-12T09:53:55.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:55 smithi111 ceph-mon[102565]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 163 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2023-12-12T09:53:57.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:57 smithi042 ceph-mon[98894]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-12-12T09:53:57.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:57 smithi173 ceph-mon[103308]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-12-12T09:53:57.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:57 smithi111 ceph-mon[102565]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2023-12-12T09:53:58.695 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:58 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-12T09:53:58.695 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:58 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:58 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-12T09:53:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:58 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:58.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:58 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-12T09:53:58.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:58 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:53:59.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:59 smithi173 ceph-mon[103308]: Deploying daemon osd.6 on smithi042 2023-12-12T09:53:59.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:53:59 smithi173 ceph-mon[103308]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-12T09:53:59.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:59 smithi042 ceph-mon[98894]: Deploying daemon osd.6 on smithi042 2023-12-12T09:53:59.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:53:59 smithi042 ceph-mon[98894]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-12T09:53:59.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:59 smithi111 ceph-mon[102565]: Deploying daemon osd.6 on smithi042 2023-12-12T09:53:59.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:53:59 smithi111 ceph-mon[102565]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-12T09:54:01.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:01 smithi173 ceph-mon[103308]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-12-12T09:54:01.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:01 smithi042 ceph-mon[98894]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-12-12T09:54:01.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:01 smithi111 ceph-mon[102565]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2023-12-12T09:54:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:03 smithi042 ceph-mon[98894]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-12-12T09:54:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:03 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:03 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:03 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:03.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:03 smithi173 ceph-mon[103308]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-12-12T09:54:03.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:03 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:03 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:03 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:03 smithi111 ceph-mon[102565]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2023-12-12T09:54:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:03 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:03 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:03 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:05.283 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 6 on host 'smithi042' 2023-12-12T09:54:05.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.280+0000 7f88c9ffb700 1 -- 172.21.15.42:0/2133985985 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f88cc069f10 con 0x7f88bc0a1500 2023-12-12T09:54:05.285 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f88bc0a1500 msgr2=0x7f88bc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:05.285 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f88bc0a1500 0x7f88bc0a39c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f88cc101570 tx=0x7f88c0005f90 comp rx=0 tx=0).stop 2023-12-12T09:54:05.285 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc1020b0 msgr2=0x7f88cc1a0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc1020b0 0x7f88cc1a0c70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f88c4007f10 tx=0x7f88c400d3b0 comp rx=0 tx=0).stop 2023-12-12T09:54:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 shutdown_connections 2023-12-12T09:54:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f88bc0a1500 0x7f88bc0a39c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f88cc100510 0x7f88cc199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88cc1020b0 0x7f88cc1a0c70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 --2- 172.21.15.42:0/2133985985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88cc101710 0x7f88cc199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 >> 172.21.15.42:0/2133985985 conn(0x7f88cc0fbad0 msgr2=0x7f88cc069800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 shutdown_connections 2023-12-12T09:54:05.286 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:05.283+0000 7f88d2aae700 1 -- 172.21.15.42:0/2133985985 wait complete. 2023-12-12T09:54:05.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:05 smithi042 ceph-mon[98894]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:54:05.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:05.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:05.801 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.802 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.802 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.802 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:05 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:05.803 DEBUG:teuthology.orchestra.run.smithi042:osd.6> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.6.service 2023-12-12T09:54:05.805 INFO:tasks.cephadm:Deploying osd.7 on smithi111 with /dev/vg_nvme/lv_2... 2023-12-12T09:54:05.805 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-12T09:54:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:05 smithi173 ceph-mon[103308]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:54:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:05 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:05.835 INFO:journalctl@ceph.osd.6.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:54:05.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:05 smithi111 ceph-mon[102565]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:54:05.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:05.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:05.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.836 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.837 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:05.837 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:05 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:06.406 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T09:54:06.523 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 09:54:06 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T09:54:06.418+0000 7fae142c4640 -1 osd.6 0 log_to_monitors true 2023-12-12T09:54:06.795 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:06 smithi042 ceph-mon[98894]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-12T09:54:06.817 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:06 smithi111 ceph-mon[102565]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-12T09:54:06.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:06 smithi173 ceph-mon[103308]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-12T09:54:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:07 smithi111 ceph-mon[102565]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:54:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:07 smithi111 ceph-mon[102565]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-12T09:54:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:07 smithi111 ceph-mon[102565]: osdmap e43: 7 total, 6 up, 7 in 2023-12-12T09:54:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:07 smithi111 ceph-mon[102565]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:54:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:07 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:07.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:07 smithi173 ceph-mon[103308]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:54:07.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:07 smithi173 ceph-mon[103308]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-12T09:54:07.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:07 smithi173 ceph-mon[103308]: osdmap e43: 7 total, 6 up, 7 in 2023-12-12T09:54:07.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:07 smithi173 ceph-mon[103308]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:54:07.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:07 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:07 smithi042 ceph-mon[98894]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:54:07.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:07 smithi042 ceph-mon[98894]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-12T09:54:07.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:07 smithi042 ceph-mon[98894]: osdmap e43: 7 total, 6 up, 7 in 2023-12-12T09:54:07.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:07 smithi042 ceph-mon[98894]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:54:07.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:07 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:07.904 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 09:54:07 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T09:54:07.534+0000 7fae0993b700 -1 osd.6 0 waiting for initial osdmap 2023-12-12T09:54:07.904 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 09:54:07 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T09:54:07.544+0000 7fae03731700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:54:08.100 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-12T09:54:08.101 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-12T09:54:08.101 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2023-12-12T09:54:08.101 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2023-12-12T09:54:08.101 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.010936 s, 959 MB/s 2023-12-12T09:54:08.101 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2023-12-12T09:54:08.468 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi111:vg_nvme/lv_2 2023-12-12T09:54:08.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:08 smithi111 ceph-mon[102565]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:54:08.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:08 smithi111 ceph-mon[102565]: osdmap e44: 7 total, 6 up, 7 in 2023-12-12T09:54:08.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:08.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:08.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:08.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:08 smithi173 ceph-mon[103308]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:54:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:08 smithi173 ceph-mon[103308]: osdmap e44: 7 total, 6 up, 7 in 2023-12-12T09:54:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:08 smithi042 ceph-mon[98894]: from='osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:54:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:08 smithi042 ceph-mon[98894]: osdmap e44: 7 total, 6 up, 7 in 2023-12-12T09:54:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:09.106 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T09:54:09.629 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi042 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] boot 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: osdmap e45: 7 total, 7 up, 7 in 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi042 to 5724M 2023-12-12T09:54:09.630 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:09.631 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:09.631 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:09.631 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:09 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:09.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:54:09.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:54:09.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:54:09.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi042 2023-12-12T09:54:09.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] boot 2023-12-12T09:54:09.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: osdmap e45: 7 total, 7 up, 7 in 2023-12-12T09:54:09.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:09.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:09.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:09.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:09.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:09.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi042 to 5724M 2023-12-12T09:54:09.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:09.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:09.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:09.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:09 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:54:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:54:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2023-12-12T09:54:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi042 2023-12-12T09:54:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: osd.6 [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] boot 2023-12-12T09:54:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: osdmap e45: 7 total, 7 up, 7 in 2023-12-12T09:54:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:54:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:09.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:09.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:09.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:09.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi042 to 5724M 2023-12-12T09:54:09.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:09.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:09.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:09.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:09 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.286+0000 7f94bc085700 1 -- 172.21.15.111:0/2474388952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101120 msgr2=0x7f94b41089c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.286+0000 7f94bc085700 1 --2- 172.21.15.111:0/2474388952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101120 0x7f94b41089c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00b0a0 tx=0x7f94ac00b3b0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.286+0000 7f94bc085700 1 -- 172.21.15.111:0/2474388952 shutdown_connections 2023-12-12T09:54:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.286+0000 7f94bc085700 1 --2- 172.21.15.111:0/2474388952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101120 0x7f94b41089c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.286+0000 7f94bc085700 1 --2- 172.21.15.111:0/2474388952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94b4101b90 0x7f94b4101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.286+0000 7f94bc085700 1 --2- 172.21.15.111:0/2474388952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94b4100760 0x7f94b4100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.286+0000 7f94bc085700 1 -- 172.21.15.111:0/2474388952 >> 172.21.15.111:0/2474388952 conn(0x7f94b40fb230 msgr2=0x7f94b40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.286+0000 7f94bc085700 1 -- 172.21.15.111:0/2474388952 shutdown_connections 2023-12-12T09:54:10.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.286+0000 7f94bc085700 1 -- 172.21.15.111:0/2474388952 wait complete. 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94bc085700 1 Processor -- start 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94bc085700 1 -- start start 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94bc085700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94b4100760 0x7f94b41964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94bc085700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94b4101120 0x7f94b4196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94bc085700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101b90 0x7f94b419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94bc085700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b419e020 con 0x7f94b4101120 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94bc085700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b419e160 con 0x7f94b4101b90 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94bc085700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b419e2f0 con 0x7f94b4100760 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94ba622700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101b90 0x7f94b419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94ba622700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101b90 0x7f94b419da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58230/0 (socket says 172.21.15.111:58230) 2023-12-12T09:54:10.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94ba622700 1 -- 172.21.15.111:0/1719232390 learned_addr learned my addr 172.21.15.111:0/1719232390 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-12T09:54:10.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94b9e21700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94b4100760 0x7f94b41964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94b9620700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94b4101120 0x7f94b4196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94ba622700 1 -- 172.21.15.111:0/1719232390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94b4100760 msgr2=0x7f94b41964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:10.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94ba622700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94b4100760 0x7f94b41964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94ba622700 1 -- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94b4101120 msgr2=0x7f94b4196a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:10.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94ba622700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94b4101120 0x7f94b4196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:10.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94ba622700 1 -- 172.21.15.111:0/1719232390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac009d00 con 0x7f94b4101b90 2023-12-12T09:54:10.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94b9e21700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94b4100760 0x7f94b41964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:54:10.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94ba622700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101b90 0x7f94b419da50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f94ac004d00 tx=0x7f94ac004de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.287+0000 7f94aaffd700 1 -- 172.21.15.111:0/1719232390 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac00e040 con 0x7f94b4101b90 2023-12-12T09:54:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.288+0000 7f94aaffd700 1 -- 172.21.15.111:0/1719232390 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f94ac01db50 con 0x7f94b4101b90 2023-12-12T09:54:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.288+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b419e570 con 0x7f94b4101b90 2023-12-12T09:54:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.288+0000 7f94aaffd700 1 -- 172.21.15.111:0/1719232390 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac01bc10 con 0x7f94b4101b90 2023-12-12T09:54:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.288+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b419e9e0 con 0x7f94b4101b90 2023-12-12T09:54:10.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.288+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b4107950 con 0x7f94b4101b90 2023-12-12T09:54:10.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.290+0000 7f94aaffd700 1 -- 172.21.15.111:0/1719232390 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f94ac01dcc0 con 0x7f94b4101b90 2023-12-12T09:54:10.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.290+0000 7f94aaffd700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f94a00a1480 0x7f94a00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:10.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.291+0000 7f94aaffd700 1 -- 172.21.15.111:0/1719232390 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f94ac0ce130 con 0x7f94b4101b90 2023-12-12T09:54:10.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.291+0000 7f94b9e21700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f94a00a1480 0x7f94a00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:10.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.291+0000 7f94b9e21700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f94a00a1480 0x7f94a00a3940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94b0005950 tx=0x7f94b00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:10.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.294+0000 7f94aaffd700 1 -- 172.21.15.111:0/1719232390 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f94ac098530 con 0x7f94b4101b90 2023-12-12T09:54:10.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:10.490+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f94b402d380 con 0x7f94a00a1480 2023-12-12T09:54:10.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:10 smithi173 ceph-mon[103308]: osdmap e46: 7 total, 7 up, 7 in 2023-12-12T09:54:10.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:54:10.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:54:10.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:10 smithi042 ceph-mon[98894]: osdmap e46: 7 total, 7 up, 7 in 2023-12-12T09:54:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:54:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:54:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:10 smithi111 ceph-mon[102565]: osdmap e46: 7 total, 7 up, 7 in 2023-12-12T09:54:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:54:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:54:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:11.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:11 smithi173 ceph-mon[103308]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:11.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:11 smithi173 ceph-mon[103308]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:54:11.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:11 smithi042 ceph-mon[98894]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:11.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:11 smithi042 ceph-mon[98894]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:54:11.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:11 smithi111 ceph-mon[102565]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:11.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:11 smithi111 ceph-mon[102565]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:54:13.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:13 smithi173 ceph-mon[103308]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:13.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:13 smithi173 ceph-mon[103308]: from='client.? 172.21.15.111:0/1798794165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c52f12-79ac-43b9-81af-d02bb17b812f"}]: dispatch 2023-12-12T09:54:13.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:13 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c52f12-79ac-43b9-81af-d02bb17b812f"}]: dispatch 2023-12-12T09:54:13.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:13 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c52f12-79ac-43b9-81af-d02bb17b812f"}]': finished 2023-12-12T09:54:13.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:13 smithi173 ceph-mon[103308]: osdmap e47: 8 total, 7 up, 8 in 2023-12-12T09:54:13.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:13 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:13.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:13 smithi173 ceph-mon[103308]: from='client.? 172.21.15.111:0/800411350' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:54:13.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:13 smithi042 ceph-mon[98894]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:13.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:13 smithi042 ceph-mon[98894]: from='client.? 172.21.15.111:0/1798794165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c52f12-79ac-43b9-81af-d02bb17b812f"}]: dispatch 2023-12-12T09:54:13.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:13 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c52f12-79ac-43b9-81af-d02bb17b812f"}]: dispatch 2023-12-12T09:54:13.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:13 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c52f12-79ac-43b9-81af-d02bb17b812f"}]': finished 2023-12-12T09:54:13.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:13 smithi042 ceph-mon[98894]: osdmap e47: 8 total, 7 up, 8 in 2023-12-12T09:54:13.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:13 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:13.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:13 smithi042 ceph-mon[98894]: from='client.? 172.21.15.111:0/800411350' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:54:13.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:13 smithi111 ceph-mon[102565]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:13.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:13 smithi111 ceph-mon[102565]: from='client.? 172.21.15.111:0/1798794165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c52f12-79ac-43b9-81af-d02bb17b812f"}]: dispatch 2023-12-12T09:54:13.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:13 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8c52f12-79ac-43b9-81af-d02bb17b812f"}]: dispatch 2023-12-12T09:54:13.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:13 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8c52f12-79ac-43b9-81af-d02bb17b812f"}]': finished 2023-12-12T09:54:13.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:13 smithi111 ceph-mon[102565]: osdmap e47: 8 total, 7 up, 8 in 2023-12-12T09:54:13.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:13 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:13.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:13 smithi111 ceph-mon[102565]: from='client.? 172.21.15.111:0/800411350' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:54:15.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:15 smithi173 ceph-mon[103308]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:15.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:15 smithi042 ceph-mon[98894]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:15.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:15 smithi111 ceph-mon[102565]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:17.818 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:17 smithi111 ceph-mon[102565]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:17.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:17 smithi173 ceph-mon[103308]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:17.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:17 smithi042 ceph-mon[98894]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:19.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:19 smithi173 ceph-mon[103308]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-12T09:54:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:19.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:19 smithi042 ceph-mon[98894]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:19.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-12T09:54:19.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:19 smithi111 ceph-mon[102565]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-12T09:54:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:20.893 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:20 smithi111 ceph-mon[102565]: Deploying daemon osd.7 on smithi111 2023-12-12T09:54:20.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:20 smithi042 ceph-mon[98894]: Deploying daemon osd.7 on smithi111 2023-12-12T09:54:21.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:20 smithi173 ceph-mon[103308]: Deploying daemon osd.7 on smithi111 2023-12-12T09:54:21.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:21 smithi042 ceph-mon[98894]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:21.931 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:21 smithi111 ceph-mon[102565]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:22.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:21 smithi173 ceph-mon[103308]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:23.895 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:23 smithi111 ceph-mon[102565]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:23.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:23 smithi042 ceph-mon[98894]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:24.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:23 smithi173 ceph-mon[103308]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:25.069 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:24 smithi111 ceph-mon[102565]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:25.069 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:24 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:25.069 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:24 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:25.069 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:24 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:25.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:24 smithi173 ceph-mon[103308]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:25.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:24 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:24 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:24 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:24 smithi042 ceph-mon[98894]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:24 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:24 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:25.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:24 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:26.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.708+0000 7f94aaffd700 1 -- 172.21.15.111:0/1719232390 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f94b402d380 con 0x7f94a00a1480 2023-12-12T09:54:26.711 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 7 on host 'smithi111' 2023-12-12T09:54:26.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.710+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f94a00a1480 msgr2=0x7f94a00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.710+0000 7f94bc085700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f94a00a1480 0x7f94a00a3940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f94b0005950 tx=0x7f94b00058e0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.710+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101b90 msgr2=0x7f94b419da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.710+0000 7f94bc085700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101b90 0x7f94b419da50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f94ac004d00 tx=0x7f94ac004de0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.710+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 shutdown_connections 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.711+0000 7f94bc085700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f94a00a1480 0x7f94a00a3940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.711+0000 7f94bc085700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f94b4101b90 0x7f94b419da50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.711+0000 7f94bc085700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94b4100760 0x7f94b41964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.711+0000 7f94bc085700 1 --2- 172.21.15.111:0/1719232390 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94b4101120 0x7f94b4196a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.711+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 >> 172.21.15.111:0/1719232390 conn(0x7f94b40fb230 msgr2=0x7f94b4104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.711+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 shutdown_connections 2023-12-12T09:54:26.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:54:26.711+0000 7f94bc085700 1 -- 172.21.15.111:0/1719232390 wait complete. 2023-12-12T09:54:27.099 DEBUG:teuthology.orchestra.run.smithi111:osd.7> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.7.service 2023-12-12T09:54:27.102 INFO:tasks.cephadm:Deploying osd.8 on smithi173 with /dev/vg_nvme/lv_2... 2023-12-12T09:54:27.102 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-12T09:54:27.138 INFO:journalctl@ceph.osd.7.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:54:27.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-mon[102565]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:27.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:27.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:27.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:27.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:27 smithi173 ceph-mon[103308]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:27.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:27.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:27.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:27 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:27 smithi042 ceph-mon[98894]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:27 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:27.711 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:54:27.871 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 09:54:27 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T09:54:27.853+0000 7f78f5e82640 -1 osd.7 0 log_to_monitors true 2023-12-12T09:54:28.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:28 smithi173 ceph-mon[103308]: from='osd.7 [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-12T09:54:28.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:28 smithi173 ceph-mon[103308]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-12T09:54:28.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:28 smithi042 ceph-mon[98894]: from='osd.7 [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-12T09:54:28.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:28 smithi042 ceph-mon[98894]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-12T09:54:28.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:28 smithi111 ceph-mon[102565]: from='osd.7 [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-12T09:54:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:28 smithi111 ceph-mon[102565]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-12T09:54:29.243 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-12T09:54:29.243 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-12T09:54:29.243 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2023-12-12T09:54:29.243 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2023-12-12T09:54:29.243 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111752 s, 938 MB/s 2023-12-12T09:54:29.243 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2023-12-12T09:54:29.576 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:29 smithi173 ceph-mon[103308]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:29.576 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:29 smithi173 ceph-mon[103308]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-12T09:54:29.576 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:29 smithi173 ceph-mon[103308]: from='osd.7 [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:54:29.576 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:29 smithi173 ceph-mon[103308]: osdmap e48: 8 total, 7 up, 8 in 2023-12-12T09:54:29.576 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:29.576 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:29 smithi173 ceph-mon[103308]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:54:29.621 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 09:54:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T09:54:29.259+0000 7f78eb4f9700 -1 osd.7 0 waiting for initial osdmap 2023-12-12T09:54:29.622 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 09:54:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T09:54:29.271+0000 7f78e72f3700 -1 osd.7 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:54:29.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:29 smithi111 ceph-mon[102565]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:29.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:29 smithi111 ceph-mon[102565]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-12T09:54:29.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:29 smithi111 ceph-mon[102565]: from='osd.7 [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:54:29.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:29 smithi111 ceph-mon[102565]: osdmap e48: 8 total, 7 up, 8 in 2023-12-12T09:54:29.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:29.622 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:29 smithi111 ceph-mon[102565]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:54:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:29 smithi042 ceph-mon[98894]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:29 smithi042 ceph-mon[98894]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-12T09:54:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:29 smithi042 ceph-mon[98894]: from='osd.7 [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:54:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:29 smithi042 ceph-mon[98894]: osdmap e48: 8 total, 7 up, 8 in 2023-12-12T09:54:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:29 smithi042 ceph-mon[98894]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:54:29.663 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi173:vg_nvme/lv_2 2023-12-12T09:54:30.258 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: osdmap e49: 8 total, 7 up, 8 in 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.523 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:30.524 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:30.524 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:30 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:54:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: osdmap e49: 8 total, 7 up, 8 in 2023-12-12T09:54:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:30.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:30.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:30.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:30.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:30.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:30.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:30.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:30 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:54:30.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: osdmap e49: 8 total, 7 up, 8 in 2023-12-12T09:54:30.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:30.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:30.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.751 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:30.751 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:30.751 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:30.751 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:30.751 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:30.751 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:30.751 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:30 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:31.481 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.477+0000 7f40bff0e700 1 -- 172.21.15.173:0/2319744287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 msgr2=0x7f40b81012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:31.481 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.477+0000 7f40bff0e700 1 --2- 172.21.15.173:0/2319744287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 0x7f40b81012f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f40b4009990 tx=0x7f40b4009ca0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.481 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.477+0000 7f40bff0e700 1 -- 172.21.15.173:0/2319744287 shutdown_connections 2023-12-12T09:54:31.481 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.477+0000 7f40bff0e700 1 --2- 172.21.15.173:0/2319744287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40b8101830 0x7f40b81080e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.481 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.477+0000 7f40bff0e700 1 --2- 172.21.15.173:0/2319744287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b80ffc90 0x7f40b81000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.481 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.477+0000 7f40bff0e700 1 --2- 172.21.15.173:0/2319744287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 0x7f40b81012f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.481 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.477+0000 7f40bff0e700 1 -- 172.21.15.173:0/2319744287 >> 172.21.15.173:0/2319744287 conn(0x7f40b80fb230 msgr2=0x7f40b80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.477+0000 7f40bff0e700 1 -- 172.21.15.173:0/2319744287 shutdown_connections 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.477+0000 7f40bff0e700 1 -- 172.21.15.173:0/2319744287 wait complete. 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bff0e700 1 Processor -- start 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bff0e700 1 -- start start 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bff0e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40b80ffc90 0x7f40b81974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bff0e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 0x7f40b8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bff0e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b8101830 0x7f40b819ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bff0e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b819f020 con 0x7f40b8101830 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bff0e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b819f160 con 0x7f40b80ffc90 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bff0e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b819f2f0 con 0x7f40b8100e90 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bd4a9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 0x7f40b8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bd4a9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 0x7f40b8197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42710/0 (socket says 172.21.15.173:42710) 2023-12-12T09:54:31.482 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bd4a9700 1 -- 172.21.15.173:0/4031926299 learned_addr learned my addr 172.21.15.173:0/4031926299 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bdcaa700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40b80ffc90 0x7f40b81974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40be4ab700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b8101830 0x7f40b819ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bd4a9700 1 -- 172.21.15.173:0/4031926299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40b80ffc90 msgr2=0x7f40b81974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bd4a9700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40b80ffc90 0x7f40b81974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bd4a9700 1 -- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b8101830 msgr2=0x7f40b819ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bd4a9700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b8101830 0x7f40b819ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bd4a9700 1 -- 172.21.15.173:0/4031926299 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b40095f0 con 0x7f40b8100e90 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.478+0000 7f40bd4a9700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 0x7f40b8197a20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f40b400b5c0 tx=0x7f40b400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.479+0000 7f40aeffd700 1 -- 172.21.15.173:0/4031926299 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b401d070 con 0x7f40b8100e90 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.479+0000 7f40aeffd700 1 -- 172.21.15.173:0/4031926299 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40b4003960 con 0x7f40b8100e90 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.479+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b819f570 con 0x7f40b8100e90 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.479+0000 7f40aeffd700 1 -- 172.21.15.173:0/4031926299 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b40056e0 con 0x7f40b8100e90 2023-12-12T09:54:31.483 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.479+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b819f980 con 0x7f40b8100e90 2023-12-12T09:54:31.484 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.479+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b8107140 con 0x7f40b8100e90 2023-12-12T09:54:31.484 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.481+0000 7f40aeffd700 1 -- 172.21.15.173:0/4031926299 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f40b4003ad0 con 0x7f40b8100e90 2023-12-12T09:54:31.484 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.481+0000 7f40aeffd700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f40a40a1480 0x7f40a40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:31.485 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.481+0000 7f40aeffd700 1 -- 172.21.15.173:0/4031926299 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5426+0+0 (secure 0 0 0) 0x7f40b40cd410 con 0x7f40b8100e90 2023-12-12T09:54:31.485 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.481+0000 7f40bdcaa700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f40a40a1480 0x7f40a40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:31.485 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.482+0000 7f40bdcaa700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f40a40a1480 0x7f40a40a3940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40b8100cf0 tx=0x7f40a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:31.486 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.485+0000 7f40aeffd700 1 -- 172.21.15.173:0/4031926299 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f40b4097640 con 0x7f40b8100e90 2023-12-12T09:54:31.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:31 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:54:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:31 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:54:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:31 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi111 2023-12-12T09:54:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:31 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi111 to 5722M 2023-12-12T09:54:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:31 smithi173 ceph-mon[103308]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:31 smithi173 ceph-mon[103308]: osd.7 [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] boot 2023-12-12T09:54:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:31 smithi173 ceph-mon[103308]: osdmap e50: 8 total, 8 up, 8 in 2023-12-12T09:54:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:31 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:54:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:31 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:54:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:31 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi111 2023-12-12T09:54:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:31 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi111 to 5722M 2023-12-12T09:54:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:31 smithi042 ceph-mon[98894]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:31 smithi042 ceph-mon[98894]: osd.7 [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] boot 2023-12-12T09:54:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:31 smithi042 ceph-mon[98894]: osdmap e50: 8 total, 8 up, 8 in 2023-12-12T09:54:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:31.684 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:31.682+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f40b802d380 con 0x7f40a40a1480 2023-12-12T09:54:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:31 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:54:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:31 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:54:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:31 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi111 2023-12-12T09:54:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:31 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi111 to 5722M 2023-12-12T09:54:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:31 smithi111 ceph-mon[102565]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 626 GiB / 626 GiB avail 2023-12-12T09:54:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:31 smithi111 ceph-mon[102565]: osd.7 [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] boot 2023-12-12T09:54:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:31 smithi111 ceph-mon[102565]: osdmap e50: 8 total, 8 up, 8 in 2023-12-12T09:54:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:54:32.526 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:32 smithi173 ceph-mon[103308]: osdmap e51: 8 total, 8 up, 8 in 2023-12-12T09:54:32.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:54:32.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:54:32.527 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:32 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:32.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:32 smithi042 ceph-mon[98894]: osdmap e51: 8 total, 8 up, 8 in 2023-12-12T09:54:32.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:54:32.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:54:32.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:32 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:32.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:32 smithi111 ceph-mon[102565]: osdmap e51: 8 total, 8 up, 8 in 2023-12-12T09:54:32.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:54:32.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:54:32.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:32 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:33 smithi173 ceph-mon[103308]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:54:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:33 smithi173 ceph-mon[103308]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:33 smithi173 ceph-mon[103308]: osdmap e52: 8 total, 8 up, 8 in 2023-12-12T09:54:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:33 smithi173 ceph-mon[103308]: osdmap e53: 8 total, 8 up, 8 in 2023-12-12T09:54:33.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:33 smithi042 ceph-mon[98894]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:54:33.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:33 smithi042 ceph-mon[98894]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:33.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:33 smithi042 ceph-mon[98894]: osdmap e52: 8 total, 8 up, 8 in 2023-12-12T09:54:33.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:33 smithi042 ceph-mon[98894]: osdmap e53: 8 total, 8 up, 8 in 2023-12-12T09:54:33.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:33 smithi111 ceph-mon[102565]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:54:33.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:33 smithi111 ceph-mon[102565]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 216 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:33.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:33 smithi111 ceph-mon[102565]: osdmap e52: 8 total, 8 up, 8 in 2023-12-12T09:54:33.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:33 smithi111 ceph-mon[102565]: osdmap e53: 8 total, 8 up, 8 in 2023-12-12T09:54:34.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:34 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/2593213070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1e8185-cda2-4a3c-9ff2-fc2480152394"}]: dispatch 2023-12-12T09:54:34.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:34 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1e8185-cda2-4a3c-9ff2-fc2480152394"}]: dispatch 2023-12-12T09:54:34.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:34 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e1e8185-cda2-4a3c-9ff2-fc2480152394"}]': finished 2023-12-12T09:54:34.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:34 smithi173 ceph-mon[103308]: osdmap e54: 9 total, 8 up, 9 in 2023-12-12T09:54:34.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:34.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:34 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/2593213070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1e8185-cda2-4a3c-9ff2-fc2480152394"}]: dispatch 2023-12-12T09:54:34.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:34 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1e8185-cda2-4a3c-9ff2-fc2480152394"}]: dispatch 2023-12-12T09:54:34.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:34 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e1e8185-cda2-4a3c-9ff2-fc2480152394"}]': finished 2023-12-12T09:54:34.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:34 smithi042 ceph-mon[98894]: osdmap e54: 9 total, 8 up, 9 in 2023-12-12T09:54:34.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:34.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:34 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/2593213070' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1e8185-cda2-4a3c-9ff2-fc2480152394"}]: dispatch 2023-12-12T09:54:34.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:34 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e1e8185-cda2-4a3c-9ff2-fc2480152394"}]: dispatch 2023-12-12T09:54:34.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:34 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e1e8185-cda2-4a3c-9ff2-fc2480152394"}]': finished 2023-12-12T09:54:34.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:34 smithi111 ceph-mon[102565]: osdmap e54: 9 total, 8 up, 9 in 2023-12-12T09:54:34.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:35.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:35 smithi173 ceph-mon[103308]: pgmap v166: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:35.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:35 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/1774420281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:54:35.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:35 smithi042 ceph-mon[98894]: pgmap v166: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:35.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:35 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/1774420281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:54:35.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:35 smithi111 ceph-mon[102565]: pgmap v166: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:35.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:35 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/1774420281' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:54:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:37 smithi173 ceph-mon[103308]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:37 smithi042 ceph-mon[98894]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:37 smithi111 ceph-mon[102565]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:39.543 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:39 smithi173 ceph-mon[103308]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:39 smithi042 ceph-mon[98894]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:39 smithi111 ceph-mon[102565]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:41.536 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:41 smithi173 ceph-mon[103308]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:41.536 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-12T09:54:41.536 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:41 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:41.536 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:41 smithi173 ceph-mon[103308]: Deploying daemon osd.8 on smithi173 2023-12-12T09:54:41.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:41 smithi042 ceph-mon[98894]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:41.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-12T09:54:41.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:41 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:41.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:41 smithi042 ceph-mon[98894]: Deploying daemon osd.8 on smithi173 2023-12-12T09:54:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:41 smithi111 ceph-mon[102565]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-12T09:54:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:41 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:41 smithi111 ceph-mon[102565]: Deploying daemon osd.8 on smithi173 2023-12-12T09:54:43.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:43 smithi173 ceph-mon[103308]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:43.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:43 smithi042 ceph-mon[98894]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:43 smithi111 ceph-mon[102565]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:45.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:45 smithi173 ceph-mon[103308]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:45.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:45 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:45.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:45 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:45.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:45 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:45 smithi042 ceph-mon[98894]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:45 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:45 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:45 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:45 smithi111 ceph-mon[102565]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:45 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:45 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:45 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:47.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:47 smithi173 ceph-mon[103308]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:47.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:47 smithi042 ceph-mon[98894]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:47 smithi111 ceph-mon[102565]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:47.998 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.996+0000 7f40aeffd700 1 -- 172.21.15.173:0/4031926299 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f40b802d380 con 0x7f40a40a1480 2023-12-12T09:54:47.998 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 8 on host 'smithi173' 2023-12-12T09:54:48.000 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f40a40a1480 msgr2=0x7f40a40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:48.000 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f40a40a1480 0x7f40a40a3940 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f40b8100cf0 tx=0x7f40a800b410 comp rx=0 tx=0).stop 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 msgr2=0x7f40b8197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 0x7f40b8197a20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f40b400b5c0 tx=0x7f40b400f690 comp rx=0 tx=0).stop 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 shutdown_connections 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f40a40a1480 0x7f40a40a3940 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40b80ffc90 0x7f40b81974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40b8101830 0x7f40b819ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 --2- 172.21.15.173:0/4031926299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f40b8100e90 0x7f40b8197a20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 >> 172.21.15.173:0/4031926299 conn(0x7f40b80fb230 msgr2=0x7f40b8104280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 shutdown_connections 2023-12-12T09:54:48.001 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:54:47.998+0000 7f40bff0e700 1 -- 172.21.15.173:0/4031926299 wait complete. 2023-12-12T09:54:48.460 DEBUG:teuthology.orchestra.run.smithi173:osd.8> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.8.service 2023-12-12T09:54:48.462 INFO:tasks.cephadm:Deploying osd.9 on smithi042 with /dev/vg_nvme/lv_1... 2023-12-12T09:54:48.463 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-12T09:54:48.492 INFO:journalctl@ceph.osd.8.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:54:48.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:48 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:48.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:48.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:48 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:48.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:48.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:48.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:54:48.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:48 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:49.084 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:54:49.227 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 09:54:49 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T09:54:49.094+0000 7f1dac1bb640 -1 osd.8 0 log_to_monitors true 2023-12-12T09:54:49.759 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:49 smithi173 ceph-mon[103308]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:49.759 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:49 smithi173 ceph-mon[103308]: from='osd.8 [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-12T09:54:49.760 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:49 smithi173 ceph-mon[103308]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-12T09:54:49.868 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:49 smithi042 ceph-mon[98894]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:49.868 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:49 smithi042 ceph-mon[98894]: from='osd.8 [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-12T09:54:49.868 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:49 smithi042 ceph-mon[98894]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-12T09:54:49.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:49 smithi111 ceph-mon[102565]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:49.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:49 smithi111 ceph-mon[102565]: from='osd.8 [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-12T09:54:49.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:49 smithi111 ceph-mon[102565]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-12T09:54:50.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:50 smithi042 ceph-mon[98894]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-12T09:54:50.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:50 smithi042 ceph-mon[98894]: from='osd.8 [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:54:50.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:50 smithi042 ceph-mon[98894]: osdmap e55: 9 total, 8 up, 9 in 2023-12-12T09:54:50.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:50 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:50.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:50 smithi042 ceph-mon[98894]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:54:50.906 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-12T09:54:50.906 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-12T09:54:50.906 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2023-12-12T09:54:50.906 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2023-12-12T09:54:50.906 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00814352 s, 1.3 GB/s 2023-12-12T09:54:50.906 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2023-12-12T09:54:50.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:50 smithi173 ceph-mon[103308]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-12T09:54:50.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:50 smithi173 ceph-mon[103308]: from='osd.8 [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:54:50.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:50 smithi173 ceph-mon[103308]: osdmap e55: 9 total, 8 up, 9 in 2023-12-12T09:54:50.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:50 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:50.933 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:50 smithi173 ceph-mon[103308]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:54:50.933 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 09:54:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T09:54:50.581+0000 7f1da1832700 -1 osd.8 0 waiting for initial osdmap 2023-12-12T09:54:50.933 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 09:54:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T09:54:50.591+0000 7f1d9be29700 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:54:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:50 smithi111 ceph-mon[102565]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-12T09:54:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:50 smithi111 ceph-mon[102565]: from='osd.8 [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:54:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:50 smithi111 ceph-mon[102565]: osdmap e55: 9 total, 8 up, 9 in 2023-12-12T09:54:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:50 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:50 smithi111 ceph-mon[102565]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:54:51.338 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi042:vg_nvme/lv_1 2023-12-12T09:54:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:54:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: osdmap e56: 9 total, 8 up, 9 in 2023-12-12T09:54:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:51.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:51.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:51.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:51.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:51.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:51 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:51.943 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:54:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:54:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: osdmap e56: 9 total, 8 up, 9 in 2023-12-12T09:54:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:51.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:51.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:51.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:51.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:51.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:51.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:51 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:52.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:52.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:54:52.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: osdmap e56: 9 total, 8 up, 9 in 2023-12-12T09:54:52.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:54:52.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:51 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:54:52.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:52 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:54:52.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:52 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:54:52.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:52 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi173 2023-12-12T09:54:52.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:52 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi173 to 5722M 2023-12-12T09:54:52.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:52 smithi042 ceph-mon[98894]: osd.8 [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] boot 2023-12-12T09:54:52.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:52 smithi042 ceph-mon[98894]: osdmap e57: 9 total, 9 up, 9 in 2023-12-12T09:54:52.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:52 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:52 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:54:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:52 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:54:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:52 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi173 2023-12-12T09:54:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:52 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi173 to 5722M 2023-12-12T09:54:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:52 smithi111 ceph-mon[102565]: osd.8 [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] boot 2023-12-12T09:54:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:52 smithi111 ceph-mon[102565]: osdmap e57: 9 total, 9 up, 9 in 2023-12-12T09:54:52.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:52 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:52 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:54:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:52 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:54:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:52 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi173 2023-12-12T09:54:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:52 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi173 to 5722M 2023-12-12T09:54:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:52 smithi173 ceph-mon[103308]: osd.8 [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] boot 2023-12-12T09:54:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:52 smithi173 ceph-mon[103308]: osdmap e57: 9 total, 9 up, 9 in 2023-12-12T09:54:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:52 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:54:53.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.191+0000 7f8118d34700 1 -- 172.21.15.42:0/2372621706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114101610 msgr2=0x7f8114101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.191+0000 7f8118d34700 1 --2- 172.21.15.42:0/2372621706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114101610 0x7f8114101a30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f81040099c0 tx=0x7f8104009cd0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.191+0000 7f8118d34700 1 -- 172.21.15.42:0/2372621706 shutdown_connections 2023-12-12T09:54:53.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.191+0000 7f8118d34700 1 --2- 172.21.15.42:0/2372621706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f81141031b0 0x7f8114109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.191+0000 7f8118d34700 1 --2- 172.21.15.42:0/2372621706 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8114101610 0x7f8114101a30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.191+0000 7f8118d34700 1 --2- 172.21.15.42:0/2372621706 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8114102810 0x7f8114102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.191+0000 7f8118d34700 1 -- 172.21.15.42:0/2372621706 >> 172.21.15.42:0/2372621706 conn(0x7f81140fcbb0 msgr2=0x7f81140feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:54:53.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.191+0000 7f8118d34700 1 -- 172.21.15.42:0/2372621706 shutdown_connections 2023-12-12T09:54:53.194 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.191+0000 7f8118d34700 1 -- 172.21.15.42:0/2372621706 wait complete. 2023-12-12T09:54:53.195 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8118d34700 1 Processor -- start 2023-12-12T09:54:53.195 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8118d34700 1 -- start start 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8118d34700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8114101610 0x7f8114075320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8118d34700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8114102810 0x7f81140739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8118d34700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81141031b0 0x7f8114073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8118d34700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81140746a0 con 0x7f81141031b0 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8118d34700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8114074810 con 0x7f8114102810 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8118d34700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81141a15b0 con 0x7f8114101610 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8112d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81141031b0 0x7f8114073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8112d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81141031b0 0x7f8114073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32860/0 (socket says 172.21.15.42:32860) 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8112d9d700 1 -- 172.21.15.42:0/2261313554 learned_addr learned my addr 172.21.15.42:0/2261313554 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8111d9b700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8114102810 0x7f81140739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f811259c700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8114101610 0x7f8114075320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8112d9d700 1 -- 172.21.15.42:0/2261313554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8114101610 msgr2=0x7f8114075320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8112d9d700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8114101610 0x7f8114075320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8112d9d700 1 -- 172.21.15.42:0/2261313554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8114102810 msgr2=0x7f81140739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8112d9d700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8114102810 0x7f81140739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8112d9d700 1 -- 172.21.15.42:0/2261313554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81040095f0 con 0x7f81141031b0 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.192+0000 7f8112d9d700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81141031b0 0x7f8114073f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f810c00ef30 tx=0x7f810c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.193+0000 7f81037fe700 1 -- 172.21.15.42:0/2261313554 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c018070 con 0x7f81141031b0 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.193+0000 7f81037fe700 1 -- 172.21.15.42:0/2261313554 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f810c00f040 con 0x7f81141031b0 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.193+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81141a17e0 con 0x7f81141031b0 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.193+0000 7f81037fe700 1 -- 172.21.15.42:0/2261313554 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00cd00 con 0x7f81141031b0 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.193+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81141a1d30 con 0x7f81141031b0 2023-12-12T09:54:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.194+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8114108b10 con 0x7f81141031b0 2023-12-12T09:54:53.198 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.195+0000 7f81037fe700 1 -- 172.21.15.42:0/2261313554 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f810c003a00 con 0x7f81141031b0 2023-12-12T09:54:53.198 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.195+0000 7f81037fe700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80fc0a1500 0x7f80fc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:54:53.198 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.196+0000 7f811259c700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80fc0a1500 0x7f80fc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:54:53.198 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.196+0000 7f81037fe700 1 -- 172.21.15.42:0/2261313554 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f810c0cc790 con 0x7f81141031b0 2023-12-12T09:54:53.199 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.196+0000 7f811259c700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80fc0a1500 0x7f80fc0a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8104003730 tx=0x7f8104000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:54:53.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.199+0000 7f81037fe700 1 -- 172.21.15.42:0/2261313554 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f810c096810 con 0x7f81141031b0 2023-12-12T09:54:53.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:54:53.406+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8114061a10 con 0x7f80fc0a1500 2023-12-12T09:54:53.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:53 smithi042 ceph-mon[98894]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:53.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:53 smithi042 ceph-mon[98894]: osdmap e58: 9 total, 9 up, 9 in 2023-12-12T09:54:53.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:53 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:54:53.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:53 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:54:53.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:53 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:53 smithi111 ceph-mon[102565]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:53 smithi111 ceph-mon[102565]: osdmap e58: 9 total, 9 up, 9 in 2023-12-12T09:54:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:53 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:54:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:53 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:54:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:53 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:54.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:53 smithi173 ceph-mon[103308]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 217 MiB used, 715 GiB / 715 GiB avail 2023-12-12T09:54:54.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:53 smithi173 ceph-mon[103308]: osdmap e58: 9 total, 9 up, 9 in 2023-12-12T09:54:54.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:53 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:54:54.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:53 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:54:54.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:53 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:54:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:54 smithi042 ceph-mon[98894]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:54:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:54 smithi042 ceph-mon[98894]: osdmap e59: 9 total, 9 up, 9 in 2023-12-12T09:54:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:54 smithi111 ceph-mon[102565]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:54:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:54 smithi111 ceph-mon[102565]: osdmap e59: 9 total, 9 up, 9 in 2023-12-12T09:54:55.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:54 smithi173 ceph-mon[103308]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:54:55.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:54 smithi173 ceph-mon[103308]: osdmap e59: 9 total, 9 up, 9 in 2023-12-12T09:54:55.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:55 smithi042 ceph-mon[98894]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:54:55.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:55 smithi042 ceph-mon[98894]: osdmap e60: 9 total, 9 up, 9 in 2023-12-12T09:54:55.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:55 smithi111 ceph-mon[102565]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:54:55.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:55 smithi111 ceph-mon[102565]: osdmap e60: 9 total, 9 up, 9 in 2023-12-12T09:54:56.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:55 smithi173 ceph-mon[103308]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:54:56.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:55 smithi173 ceph-mon[103308]: osdmap e60: 9 total, 9 up, 9 in 2023-12-12T09:54:56.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:56 smithi042 ceph-mon[98894]: osdmap e61: 9 total, 9 up, 9 in 2023-12-12T09:54:56.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:56 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/140393348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c936a54-8440-4cbd-ba0b-79d236e99180"}]: dispatch 2023-12-12T09:54:56.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:56 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/140393348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c936a54-8440-4cbd-ba0b-79d236e99180"}]': finished 2023-12-12T09:54:56.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:56 smithi042 ceph-mon[98894]: osdmap e62: 10 total, 9 up, 10 in 2023-12-12T09:54:56.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:56 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:54:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:56 smithi111 ceph-mon[102565]: osdmap e61: 9 total, 9 up, 9 in 2023-12-12T09:54:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:56 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/140393348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c936a54-8440-4cbd-ba0b-79d236e99180"}]: dispatch 2023-12-12T09:54:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:56 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/140393348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c936a54-8440-4cbd-ba0b-79d236e99180"}]': finished 2023-12-12T09:54:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:56 smithi111 ceph-mon[102565]: osdmap e62: 10 total, 9 up, 10 in 2023-12-12T09:54:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:56 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:54:57.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:56 smithi173 ceph-mon[103308]: osdmap e61: 9 total, 9 up, 9 in 2023-12-12T09:54:57.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:56 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/140393348' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1c936a54-8440-4cbd-ba0b-79d236e99180"}]: dispatch 2023-12-12T09:54:57.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:56 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/140393348' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1c936a54-8440-4cbd-ba0b-79d236e99180"}]': finished 2023-12-12T09:54:57.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:56 smithi173 ceph-mon[103308]: osdmap e62: 10 total, 9 up, 10 in 2023-12-12T09:54:57.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:56 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:54:57.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:57 smithi042 ceph-mon[98894]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-12T09:54:57.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:57 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2984160305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:54:57.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:57 smithi111 ceph-mon[102565]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-12T09:54:57.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:57 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2984160305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:54:58.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:57 smithi173 ceph-mon[103308]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 244 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-12T09:54:58.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:57 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2984160305' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:54:59.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:54:59 smithi042 ceph-mon[98894]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-12-12T09:54:59.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:54:59 smithi111 ceph-mon[102565]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-12-12T09:55:00.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:54:59 smithi173 ceph-mon[103308]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 138 KiB/s, 0 objects/s recovering 2023-12-12T09:55:01.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:01 smithi042 ceph-mon[98894]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-12T09:55:01.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:01 smithi111 ceph-mon[102565]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-12T09:55:02.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:01 smithi173 ceph-mon[103308]: pgmap v188: 1 pgs: 1 peering; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-12T09:55:03.737 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:03 smithi042 ceph-mon[98894]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-12-12T09:55:03.737 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:03 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-12T09:55:03.738 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:03 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:03 smithi111 ceph-mon[102565]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-12-12T09:55:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:03 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-12T09:55:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:03 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:04.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:03 smithi173 ceph-mon[103308]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-12-12T09:55:04.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:03 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-12T09:55:04.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:03 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:04 smithi042 ceph-mon[98894]: Deploying daemon osd.9 on smithi042 2023-12-12T09:55:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:04 smithi111 ceph-mon[102565]: Deploying daemon osd.9 on smithi042 2023-12-12T09:55:05.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:04 smithi173 ceph-mon[103308]: Deploying daemon osd.9 on smithi042 2023-12-12T09:55:05.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:05 smithi042 ceph-mon[98894]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:05.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:05 smithi111 ceph-mon[102565]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:06.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:05 smithi173 ceph-mon[103308]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:07 smithi042 ceph-mon[98894]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:07 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:07 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:07.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:07 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:07.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:07 smithi111 ceph-mon[102565]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:07.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:07 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:07.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:07 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:07.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:07 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:08.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:07 smithi173 ceph-mon[103308]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:08.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:07 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:08.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:07 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:08.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:07 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:09 smithi042 ceph-mon[98894]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:09.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:09 smithi111 ceph-mon[102565]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:10.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:09 smithi173 ceph-mon[103308]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:10.315 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 9 on host 'smithi042' 2023-12-12T09:55:10.316 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.313+0000 7f81037fe700 1 -- 172.21.15.42:0/2261313554 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8114061a10 con 0x7f80fc0a1500 2023-12-12T09:55:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80fc0a1500 msgr2=0x7f80fc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80fc0a1500 0x7f80fc0a39c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8104003730 tx=0x7f8104000bc0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81141031b0 msgr2=0x7f8114073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81141031b0 0x7f8114073f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f810c00ef30 tx=0x7f810c00c3a0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 shutdown_connections 2023-12-12T09:55:10.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80fc0a1500 0x7f80fc0a39c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.318 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f81141031b0 0x7f8114073f00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.318 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8114101610 0x7f8114075320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.318 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 --2- 172.21.15.42:0/2261313554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8114102810 0x7f81140739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:10.318 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 >> 172.21.15.42:0/2261313554 conn(0x7f81140fcbb0 msgr2=0x7f8114105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:10.318 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 shutdown_connections 2023-12-12T09:55:10.318 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:10.315+0000 7f8118d34700 1 -- 172.21.15.42:0/2261313554 wait complete. 2023-12-12T09:55:10.745 DEBUG:teuthology.orchestra.run.smithi042:osd.9> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.9.service 2023-12-12T09:55:10.748 INFO:tasks.cephadm:Deploying osd.10 on smithi111 with /dev/vg_nvme/lv_1... 2023-12-12T09:55:10.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-12T09:55:10.780 INFO:journalctl@ceph.osd.9.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:55:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:10 smithi111 ceph-mon[102565]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:11.268 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 09:55:11 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T09:55:11.204+0000 7fa92c1eb640 -1 osd.9 0 log_to_monitors true 2023-12-12T09:55:11.268 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:10 smithi042 ceph-mon[98894]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:11.269 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.269 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.269 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:11.269 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:11.269 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.269 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.269 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.269 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:10 smithi173 ceph-mon[103308]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:11.351 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T09:55:12.139 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:12 smithi042 ceph-mon[98894]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-12T09:55:12.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:12 smithi111 ceph-mon[102565]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-12T09:55:12.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:12 smithi173 ceph-mon[103308]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi111.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi111.stderr: stderr: 10+0 records in 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi111.stderr:10+0 records out 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi111.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0110441 s, 949 MB/s 2023-12-12T09:55:12.878 INFO:teuthology.orchestra.run.smithi111.stderr:--> Zapping successful for: 2023-12-12T09:55:13.210 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:13 smithi111 ceph-mon[102565]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:13.210 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:13 smithi111 ceph-mon[102565]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-12T09:55:13.210 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:13 smithi111 ceph-mon[102565]: osdmap e63: 10 total, 9 up, 10 in 2023-12-12T09:55:13.210 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:13 smithi111 ceph-mon[102565]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:55:13.210 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:13 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:13.211 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi111:vg_nvme/lv_1 2023-12-12T09:55:13.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:13 smithi173 ceph-mon[103308]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:13 smithi173 ceph-mon[103308]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-12T09:55:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:13 smithi173 ceph-mon[103308]: osdmap e63: 10 total, 9 up, 10 in 2023-12-12T09:55:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:13 smithi173 ceph-mon[103308]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:55:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:13 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:13.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:13 smithi042 ceph-mon[98894]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:13.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:13 smithi042 ceph-mon[98894]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-12T09:55:13.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:13 smithi042 ceph-mon[98894]: osdmap e63: 10 total, 9 up, 10 in 2023-12-12T09:55:13.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:13 smithi042 ceph-mon[98894]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2023-12-12T09:55:13.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:13 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:13.404 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 09:55:13 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T09:55:13.007+0000 7fa921862700 -1 osd.9 0 waiting for initial osdmap 2023-12-12T09:55:13.404 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 09:55:13 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T09:55:13.024+0000 7fa91d65c700 -1 osd.9 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:55:13.808 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T09:55:14.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:55:14.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: osdmap e64: 10 total, 9 up, 10 in 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi042 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:14 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: osdmap e64: 10 total, 9 up, 10 in 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi042 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:14.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:14.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:14 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2023-12-12T09:55:14.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: osdmap e64: 10 total, 9 up, 10 in 2023-12-12T09:55:14.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:14.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:14.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi042 2023-12-12T09:55:14.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:14.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:14.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:14.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:14.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:14 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:15.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.091+0000 7efecfeae700 1 -- 172.21.15.111:0/1420683826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8100760 msgr2=0x7efec8100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.091+0000 7efecfeae700 1 --2- 172.21.15.111:0/1420683826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8100760 0x7efec8100be0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efec4009990 tx=0x7efec4009ca0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.091+0000 7efecfeae700 1 -- 172.21.15.111:0/1420683826 shutdown_connections 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.091+0000 7efecfeae700 1 --2- 172.21.15.111:0/1420683826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efec8101120 0x7efec81089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.091+0000 7efecfeae700 1 --2- 172.21.15.111:0/1420683826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8101b90 0x7efec8101fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.091+0000 7efecfeae700 1 --2- 172.21.15.111:0/1420683826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8100760 0x7efec8100be0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.091+0000 7efecfeae700 1 -- 172.21.15.111:0/1420683826 >> 172.21.15.111:0/1420683826 conn(0x7efec80fb230 msgr2=0x7efec80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 -- 172.21.15.111:0/1420683826 shutdown_connections 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 -- 172.21.15.111:0/1420683826 wait complete. 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 Processor -- start 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 -- start start 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efec8100760 0x7efec8196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8101120 0x7efec81969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8101b90 0x7efec819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec819dfb0 con 0x7efec8101120 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec819e0f0 con 0x7efec8101b90 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecfeae700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec80ff6d0 con 0x7efec8100760 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efece44b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8101b90 0x7efec819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efece44b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8101b90 0x7efec819d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58368/0 (socket says 172.21.15.111:58368) 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efece44b700 1 -- 172.21.15.111:0/1081318175 learned_addr learned my addr 172.21.15.111:0/1081318175 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.092+0000 7efecdc4a700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efec8100760 0x7efec8196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efecd449700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8101120 0x7efec81969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efece44b700 1 -- 172.21.15.111:0/1081318175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efec8100760 msgr2=0x7efec8196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efece44b700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efec8100760 0x7efec8196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efece44b700 1 -- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8101120 msgr2=0x7efec81969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efece44b700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8101120 0x7efec81969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efece44b700 1 -- 172.21.15.111:0/1081318175 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec40095f0 con 0x7efec8101b90 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efece44b700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8101b90 0x7efec819d9e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efec0007ee0 tx=0x7efec000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efebeffd700 1 -- 172.21.15.111:0/1081318175 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0017070 con 0x7efec8101b90 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efebeffd700 1 -- 172.21.15.111:0/1081318175 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efec000f040 con 0x7efec8101b90 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec80ff9b0 con 0x7efec8101b90 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efebeffd700 1 -- 172.21.15.111:0/1081318175 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0005740 con 0x7efec8101b90 2023-12-12T09:55:15.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.093+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec80ffeb0 con 0x7efec8101b90 2023-12-12T09:55:15.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.094+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec8107a20 con 0x7efec8101b90 2023-12-12T09:55:15.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.095+0000 7efebeffd700 1 -- 172.21.15.111:0/1081318175 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7efec000dc60 con 0x7efec8101b90 2023-12-12T09:55:15.099 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.096+0000 7efebeffd700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efeb40a1500 0x7efeb40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:15.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.096+0000 7efebeffd700 1 -- 172.21.15.111:0/1081318175 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6261+0+0 (secure 0 0 0) 0x7efec00cc3b0 con 0x7efec8101b90 2023-12-12T09:55:15.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.096+0000 7efecdc4a700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efeb40a1500 0x7efeb40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:15.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.096+0000 7efecdc4a700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efeb40a1500 0x7efeb40a39c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efeb8005950 tx=0x7efeb800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:15.101 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.099+0000 7efebeffd700 1 -- 172.21.15.111:0/1081318175 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7efec0096170 con 0x7efec8101b90 2023-12-12T09:55:15.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:15 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:55:15.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:15 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:55:15.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:15 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi042 to 4293M 2023-12-12T09:55:15.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:15 smithi111 ceph-mon[102565]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:15.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:15 smithi111 ceph-mon[102565]: osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] boot 2023-12-12T09:55:15.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:15 smithi111 ceph-mon[102565]: osdmap e65: 10 total, 10 up, 10 in 2023-12-12T09:55:15.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:15.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:15.298+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7efec81000d0 con 0x7efeb40a1500 2023-12-12T09:55:15.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:15 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:55:15.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:15 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:55:15.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:15 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi042 to 4293M 2023-12-12T09:55:15.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:15 smithi173 ceph-mon[103308]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:15.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:15 smithi173 ceph-mon[103308]: osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] boot 2023-12-12T09:55:15.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:15 smithi173 ceph-mon[103308]: osdmap e65: 10 total, 10 up, 10 in 2023-12-12T09:55:15.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:15 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:55:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:15 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:55:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:15 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi042 to 4293M 2023-12-12T09:55:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:15 smithi042 ceph-mon[98894]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 245 MiB used, 804 GiB / 805 GiB avail 2023-12-12T09:55:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:15 smithi042 ceph-mon[98894]: osd.9 [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] boot 2023-12-12T09:55:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:15 smithi042 ceph-mon[98894]: osdmap e65: 10 total, 10 up, 10 in 2023-12-12T09:55:15.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:55:16.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:16 smithi111 ceph-mon[102565]: osdmap e66: 10 total, 10 up, 10 in 2023-12-12T09:55:16.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:16 smithi111 ceph-mon[102565]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:55:16.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:16 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:55:16.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:16 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:55:16.298 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:16 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:16.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:16 smithi173 ceph-mon[103308]: osdmap e66: 10 total, 10 up, 10 in 2023-12-12T09:55:16.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:16 smithi173 ceph-mon[103308]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:55:16.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:55:16.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:55:16.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:16 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:16.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:16 smithi042 ceph-mon[98894]: osdmap e66: 10 total, 10 up, 10 in 2023-12-12T09:55:16.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:16 smithi042 ceph-mon[98894]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi111:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:55:16.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:55:16.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:55:16.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:16 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:17.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:17 smithi111 ceph-mon[102565]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:17.322 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:17 smithi173 ceph-mon[103308]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:17.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:17 smithi042 ceph-mon[98894]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:18.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:18 smithi173 ceph-mon[103308]: from='client.? 172.21.15.111:0/3294927156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963a1538-8ad6-4832-9f32-cc7bb0642555"}]: dispatch 2023-12-12T09:55:18.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:18 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963a1538-8ad6-4832-9f32-cc7bb0642555"}]: dispatch 2023-12-12T09:55:18.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:18 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "963a1538-8ad6-4832-9f32-cc7bb0642555"}]': finished 2023-12-12T09:55:18.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:18 smithi173 ceph-mon[103308]: osdmap e67: 11 total, 10 up, 11 in 2023-12-12T09:55:18.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:18 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:18.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:18 smithi042 ceph-mon[98894]: from='client.? 172.21.15.111:0/3294927156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963a1538-8ad6-4832-9f32-cc7bb0642555"}]: dispatch 2023-12-12T09:55:18.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:18 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963a1538-8ad6-4832-9f32-cc7bb0642555"}]: dispatch 2023-12-12T09:55:18.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:18 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "963a1538-8ad6-4832-9f32-cc7bb0642555"}]': finished 2023-12-12T09:55:18.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:18 smithi042 ceph-mon[98894]: osdmap e67: 11 total, 10 up, 11 in 2023-12-12T09:55:18.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:18 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:18.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:18 smithi111 ceph-mon[102565]: from='client.? 172.21.15.111:0/3294927156' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963a1538-8ad6-4832-9f32-cc7bb0642555"}]: dispatch 2023-12-12T09:55:18.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:18 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "963a1538-8ad6-4832-9f32-cc7bb0642555"}]: dispatch 2023-12-12T09:55:18.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:18 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "963a1538-8ad6-4832-9f32-cc7bb0642555"}]': finished 2023-12-12T09:55:18.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:18 smithi111 ceph-mon[102565]: osdmap e67: 11 total, 10 up, 11 in 2023-12-12T09:55:18.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:18 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:19.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:19 smithi111 ceph-mon[102565]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:19.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:19 smithi111 ceph-mon[102565]: from='client.? 172.21.15.111:0/780522902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:55:19.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:19 smithi173 ceph-mon[103308]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:19.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:19 smithi173 ceph-mon[103308]: from='client.? 172.21.15.111:0/780522902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:55:19.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:19 smithi042 ceph-mon[98894]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:19.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:19 smithi042 ceph-mon[98894]: from='client.? 172.21.15.111:0/780522902' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:55:21.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:21 smithi111 ceph-mon[102565]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:21.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:21 smithi173 ceph-mon[103308]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:21.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:21 smithi042 ceph-mon[98894]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:23.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:23 smithi111 ceph-mon[102565]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:23.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:23 smithi173 ceph-mon[103308]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:23.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:23 smithi042 ceph-mon[98894]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:24.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:24 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-12T09:55:24.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:24 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:24.648 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:24 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-12T09:55:24.649 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:24 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:24.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:24 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-12T09:55:24.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:24 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:25.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:25 smithi173 ceph-mon[103308]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:25.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:25 smithi173 ceph-mon[103308]: Deploying daemon osd.10 on smithi111 2023-12-12T09:55:25.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:25 smithi042 ceph-mon[98894]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:25.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:25 smithi042 ceph-mon[98894]: Deploying daemon osd.10 on smithi111 2023-12-12T09:55:25.686 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:25 smithi111 ceph-mon[102565]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:25.686 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:25 smithi111 ceph-mon[102565]: Deploying daemon osd.10 on smithi111 2023-12-12T09:55:27.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:27 smithi173 ceph-mon[103308]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:27.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:27 smithi042 ceph-mon[98894]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:27.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:27 smithi111 ceph-mon[102565]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:29.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:29 smithi111 ceph-mon[102565]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:29.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:29.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:29.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:29.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:29 smithi042 ceph-mon[98894]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:29.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:29.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:29.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:29.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:29 smithi173 ceph-mon[103308]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:29.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:29.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:29.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:30.766 INFO:teuthology.orchestra.run.smithi111.stdout:Created osd(s) 10 on host 'smithi111' 2023-12-12T09:55:30.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.763+0000 7efebeffd700 1 -- 172.21.15.111:0/1081318175 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7efec81000d0 con 0x7efeb40a1500 2023-12-12T09:55:30.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efeb40a1500 msgr2=0x7efeb40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:30.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efeb40a1500 0x7efeb40a39c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7efeb8005950 tx=0x7efeb800a300 comp rx=0 tx=0).stop 2023-12-12T09:55:30.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8101b90 msgr2=0x7efec819d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:30.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8101b90 0x7efec819d9e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efec0007ee0 tx=0x7efec000d3b0 comp rx=0 tx=0).stop 2023-12-12T09:55:30.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 shutdown_connections 2023-12-12T09:55:30.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efeb40a1500 0x7efeb40a39c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:30.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efec8101b90 0x7efec819d9e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:30.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efec8100760 0x7efec8196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:30.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 --2- 172.21.15.111:0/1081318175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efec8101120 0x7efec81969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:30.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 >> 172.21.15.111:0/1081318175 conn(0x7efec80fb230 msgr2=0x7efec8104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:30.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 shutdown_connections 2023-12-12T09:55:30.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-12-12T09:55:30.766+0000 7efecfeae700 1 -- 172.21.15.111:0/1081318175 wait complete. 2023-12-12T09:55:31.100 DEBUG:teuthology.orchestra.run.smithi111:osd.10> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.10.service 2023-12-12T09:55:31.102 INFO:tasks.cephadm:Deploying osd.11 on smithi173 with /dev/vg_nvme/lv_1... 2023-12-12T09:55:31.103 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-12T09:55:31.133 INFO:journalctl@ceph.osd.10.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:55:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:31 smithi042 ceph-mon[98894]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:31 smithi111 ceph-mon[102565]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:31.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:31 smithi173 ceph-mon[103308]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:31.709 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:55:32.228 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:32 smithi173 ceph-mon[103308]: from='osd.10 [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-12T09:55:32.229 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:32 smithi173 ceph-mon[103308]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-12T09:55:32.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:32 smithi111 ceph-mon[102565]: from='osd.10 [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-12T09:55:32.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:32 smithi111 ceph-mon[102565]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-12T09:55:32.247 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 09:55:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T09:55:32.107+0000 7f756a112640 -1 osd.10 0 log_to_monitors true 2023-12-12T09:55:32.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:32 smithi042 ceph-mon[98894]: from='osd.10 [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-12T09:55:32.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:32 smithi042 ceph-mon[98894]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-12T09:55:33.220 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-12T09:55:33.220 INFO:teuthology.orchestra.run.smithi173.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-12T09:55:33.221 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10+0 records in 2023-12-12T09:55:33.221 INFO:teuthology.orchestra.run.smithi173.stderr:10+0 records out 2023-12-12T09:55:33.221 INFO:teuthology.orchestra.run.smithi173.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126039 s, 832 MB/s 2023-12-12T09:55:33.221 INFO:teuthology.orchestra.run.smithi173.stderr:--> Zapping successful for: 2023-12-12T09:55:33.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:33 smithi042 ceph-mon[98894]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:33.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:33 smithi042 ceph-mon[98894]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-12T09:55:33.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:33 smithi042 ceph-mon[98894]: from='osd.10 [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:55:33.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:33 smithi042 ceph-mon[98894]: osdmap e68: 11 total, 10 up, 11 in 2023-12-12T09:55:33.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:33 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:33.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:33 smithi042 ceph-mon[98894]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:55:33.417 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:33 smithi111 ceph-mon[102565]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:33.417 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:33 smithi111 ceph-mon[102565]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-12T09:55:33.417 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:33 smithi111 ceph-mon[102565]: from='osd.10 [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:55:33.417 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:33 smithi111 ceph-mon[102565]: osdmap e68: 11 total, 10 up, 11 in 2023-12-12T09:55:33.417 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:33 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:33.418 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:33 smithi111 ceph-mon[102565]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:55:33.418 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 09:55:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T09:55:33.141+0000 7f755f789700 -1 osd.10 0 waiting for initial osdmap 2023-12-12T09:55:33.418 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 09:55:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T09:55:33.152+0000 7f755b583700 -1 osd.10 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:55:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:33 smithi173 ceph-mon[103308]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:33 smithi173 ceph-mon[103308]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-12T09:55:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:33 smithi173 ceph-mon[103308]: from='osd.10 [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:55:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:33 smithi173 ceph-mon[103308]: osdmap e68: 11 total, 10 up, 11 in 2023-12-12T09:55:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:33 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:33.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:33 smithi173 ceph-mon[103308]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]: dispatch 2023-12-12T09:55:33.672 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch daemon add osd smithi173:vg_nvme/lv_1 2023-12-12T09:55:34.271 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:55:34.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:55:34.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: osdmap e69: 11 total, 10 up, 11 in 2023-12-12T09:55:34.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:34.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:34.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: osd.10 [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] boot 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: osdmap e70: 11 total, 11 up, 11 in 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi111 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi111 to 4292M 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:34.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:34 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: osdmap e69: 11 total, 10 up, 11 in 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: osd.10 [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] boot 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: osdmap e70: 11 total, 11 up, 11 in 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi111 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.484 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi111 to 4292M 2023-12-12T09:55:34.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:34.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:34.485 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:34 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi111", "root=default"]}]': finished 2023-12-12T09:55:34.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: osdmap e69: 11 total, 10 up, 11 in 2023-12-12T09:55:34.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:34.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:34.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: osd.10 [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] boot 2023-12-12T09:55:34.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: osdmap e70: 11 total, 11 up, 11 in 2023-12-12T09:55:34.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi111 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi111 to 4292M 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:34.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:34 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:35 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:55:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:35 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:55:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:35 smithi173 ceph-mon[103308]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:35 smithi173 ceph-mon[103308]: osdmap e71: 11 total, 11 up, 11 in 2023-12-12T09:55:35.548 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.546+0000 7fd818860700 1 -- 172.21.15.173:0/1948246854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814101620 msgr2=0x7fd814101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:35.549 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.546+0000 7fd818860700 1 --2- 172.21.15.173:0/1948246854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814101620 0x7fd814101a40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd804009990 tx=0x7fd804009ca0 comp rx=0 tx=0).stop 2023-12-12T09:55:35.549 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.546+0000 7fd818860700 1 -- 172.21.15.173:0/1948246854 shutdown_connections 2023-12-12T09:55:35.549 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.546+0000 7fd818860700 1 --2- 172.21.15.173:0/1948246854 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd8141031c0 0x7fd814109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:35.549 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.546+0000 7fd818860700 1 --2- 172.21.15.173:0/1948246854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814101620 0x7fd814101a40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:35.549 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.546+0000 7fd818860700 1 --2- 172.21.15.173:0/1948246854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd814102820 0x7fd814102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:35.549 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.546+0000 7fd818860700 1 -- 172.21.15.173:0/1948246854 >> 172.21.15.173:0/1948246854 conn(0x7fd8140fcbc0 msgr2=0x7fd8140ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:35.549 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 -- 172.21.15.173:0/1948246854 shutdown_connections 2023-12-12T09:55:35.549 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 -- 172.21.15.173:0/1948246854 wait complete. 2023-12-12T09:55:35.551 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 Processor -- start 2023-12-12T09:55:35.551 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 -- start start 2023-12-12T09:55:35.551 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101620 0x7fd8141998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:35.551 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814102820 0x7fd814199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:35.551 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8141031c0 0x7fd8141a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:35.551 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141a13b0 con 0x7fd814101620 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141a14f0 con 0x7fd8141031c0 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.547+0000 7fd818860700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8141a1680 con 0x7fd814102820 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd811d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814102820 0x7fd814199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd811d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814102820 0x7fd814199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42862/0 (socket says 172.21.15.173:42862) 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd811d9b700 1 -- 172.21.15.173:0/641019429 learned_addr learned my addr 172.21.15.173:0/641019429 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd812d9d700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8141031c0 0x7fd8141a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd81259c700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101620 0x7fd8141998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd811d9b700 1 -- 172.21.15.173:0/641019429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8141031c0 msgr2=0x7fd8141a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd811d9b700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8141031c0 0x7fd8141a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd811d9b700 1 -- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101620 msgr2=0x7fd8141998c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd811d9b700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101620 0x7fd8141998c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd811d9b700 1 -- 172.21.15.173:0/641019429 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8040095f0 con 0x7fd814102820 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd81259c700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101620 0x7fd8141998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:55:35.552 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd811d9b700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814102820 0x7fd814199e00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd80800b560 tx=0x7fd80800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:35.553 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd8037fe700 1 -- 172.21.15.173:0/641019429 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808011830 con 0x7fd814102820 2023-12-12T09:55:35.553 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd8037fe700 1 -- 172.21.15.173:0/641019429 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd808011e50 con 0x7fd814102820 2023-12-12T09:55:35.553 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8141a1960 con 0x7fd814102820 2023-12-12T09:55:35.553 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd8037fe700 1 -- 172.21.15.173:0/641019429 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80800f2e0 con 0x7fd814102820 2023-12-12T09:55:35.553 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.548+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8141a1ee0 con 0x7fd814102820 2023-12-12T09:55:35.553 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.549+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81404f290 con 0x7fd814102820 2023-12-12T09:55:35.554 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.550+0000 7fd8037fe700 1 -- 172.21.15.173:0/641019429 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd808010580 con 0x7fd814102820 2023-12-12T09:55:35.554 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.551+0000 7fd8037fe700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7fc0a1500 0x7fd7fc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:35.554 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.551+0000 7fd8037fe700 1 -- 172.21.15.173:0/641019429 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fd8080cc140 con 0x7fd814102820 2023-12-12T09:55:35.554 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.551+0000 7fd81259c700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7fc0a1500 0x7fd7fc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:35.554 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.552+0000 7fd81259c700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7fc0a1500 0x7fd7fc0a39c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd8040038c0 tx=0x7fd804005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:35.557 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.555+0000 7fd8037fe700 1 -- 172.21.15.173:0/641019429 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd808095dd0 con 0x7fd814102820 2023-12-12T09:55:35.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:35 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:55:35.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:35 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:55:35.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:35 smithi042 ceph-mon[98894]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:35.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:35 smithi042 ceph-mon[98894]: osdmap e71: 11 total, 11 up, 11 in 2023-12-12T09:55:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:35 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:55:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:35 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:55:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:35 smithi111 ceph-mon[102565]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 894 GiB / 894 GiB avail 2023-12-12T09:55:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:35 smithi111 ceph-mon[102565]: osdmap e71: 11 total, 11 up, 11 in 2023-12-12T09:55:35.754 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:35.752+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd814074f20 con 0x7fd7fc0a1500 2023-12-12T09:55:36.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:36 smithi173 ceph-mon[103308]: osdmap e72: 11 total, 11 up, 11 in 2023-12-12T09:55:36.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:36 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:55:36.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:36 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:55:36.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:36 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:36.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:36 smithi042 ceph-mon[98894]: osdmap e72: 11 total, 11 up, 11 in 2023-12-12T09:55:36.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:36 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:55:36.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:36 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:55:36.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:36 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:36.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:36 smithi111 ceph-mon[102565]: osdmap e72: 11 total, 11 up, 11 in 2023-12-12T09:55:36.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:36 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-12T09:55:36.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:36 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-12T09:55:36.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:36 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:37 smithi173 ceph-mon[103308]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:55:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:37 smithi173 ceph-mon[103308]: pgmap v216: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-12-12T09:55:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:37 smithi173 ceph-mon[103308]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-12-12T09:55:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:37 smithi173 ceph-mon[103308]: osdmap e73: 11 total, 11 up, 11 in 2023-12-12T09:55:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:37 smithi042 ceph-mon[98894]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:55:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:37 smithi042 ceph-mon[98894]: pgmap v216: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-12-12T09:55:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:37 smithi042 ceph-mon[98894]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-12-12T09:55:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:37 smithi042 ceph-mon[98894]: osdmap e73: 11 total, 11 up, 11 in 2023-12-12T09:55:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:37 smithi111 ceph-mon[102565]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi173:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:55:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:37 smithi111 ceph-mon[102565]: pgmap v216: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 299 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-12-12T09:55:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:37 smithi111 ceph-mon[102565]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2023-12-12T09:55:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:37 smithi111 ceph-mon[102565]: osdmap e73: 11 total, 11 up, 11 in 2023-12-12T09:55:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:38 smithi173 ceph-mon[103308]: osdmap e74: 11 total, 11 up, 11 in 2023-12-12T09:55:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:38 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/2174273989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840f024c-8e3a-4949-8e31-27f28d498d95"}]: dispatch 2023-12-12T09:55:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:38 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840f024c-8e3a-4949-8e31-27f28d498d95"}]: dispatch 2023-12-12T09:55:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:38 smithi173 ceph-mon[103308]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "840f024c-8e3a-4949-8e31-27f28d498d95"}]': finished 2023-12-12T09:55:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:38 smithi173 ceph-mon[103308]: osdmap e75: 12 total, 11 up, 12 in 2023-12-12T09:55:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:38 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:38 smithi042 ceph-mon[98894]: osdmap e74: 11 total, 11 up, 11 in 2023-12-12T09:55:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:38 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/2174273989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840f024c-8e3a-4949-8e31-27f28d498d95"}]: dispatch 2023-12-12T09:55:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:38 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840f024c-8e3a-4949-8e31-27f28d498d95"}]: dispatch 2023-12-12T09:55:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:38 smithi042 ceph-mon[98894]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "840f024c-8e3a-4949-8e31-27f28d498d95"}]': finished 2023-12-12T09:55:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:38 smithi042 ceph-mon[98894]: osdmap e75: 12 total, 11 up, 12 in 2023-12-12T09:55:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:38 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:38 smithi111 ceph-mon[102565]: osdmap e74: 11 total, 11 up, 11 in 2023-12-12T09:55:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:38 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/2174273989' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840f024c-8e3a-4949-8e31-27f28d498d95"}]: dispatch 2023-12-12T09:55:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:38 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "840f024c-8e3a-4949-8e31-27f28d498d95"}]: dispatch 2023-12-12T09:55:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:38 smithi111 ceph-mon[102565]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "840f024c-8e3a-4949-8e31-27f28d498d95"}]': finished 2023-12-12T09:55:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:38 smithi111 ceph-mon[102565]: osdmap e75: 12 total, 11 up, 12 in 2023-12-12T09:55:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:38 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:39.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:39 smithi173 ceph-mon[103308]: pgmap v219: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-12-12T09:55:39.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:39 smithi173 ceph-mon[103308]: from='client.? 172.21.15.173:0/4282383397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:55:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:39 smithi042 ceph-mon[98894]: pgmap v219: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-12-12T09:55:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:39 smithi042 ceph-mon[98894]: from='client.? 172.21.15.173:0/4282383397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:55:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:39 smithi111 ceph-mon[102565]: pgmap v219: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-12-12T09:55:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:39 smithi111 ceph-mon[102565]: from='client.? 172.21.15.173:0/4282383397' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-12T09:55:41.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:41 smithi173 ceph-mon[103308]: pgmap v221: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-12-12T09:55:41.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:41 smithi042 ceph-mon[98894]: pgmap v221: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-12-12T09:55:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:41 smithi111 ceph-mon[102565]: pgmap v221: 1 pgs: 1 activating+undersized+degraded+remapped; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2023-12-12T09:55:42.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:42 smithi173 ceph-mon[103308]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-12-12T09:55:42.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:42 smithi173 ceph-mon[103308]: Cluster is now healthy 2023-12-12T09:55:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:42 smithi042 ceph-mon[98894]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-12-12T09:55:42.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:42 smithi042 ceph-mon[98894]: Cluster is now healthy 2023-12-12T09:55:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:42 smithi111 ceph-mon[102565]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2023-12-12T09:55:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:42 smithi111 ceph-mon[102565]: Cluster is now healthy 2023-12-12T09:55:43.418 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:43 smithi173 ceph-mon[103308]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-12T09:55:43.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:43 smithi042 ceph-mon[98894]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-12T09:55:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:43 smithi111 ceph-mon[102565]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-12T09:55:45.613 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:45 smithi173 ceph-mon[103308]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-12-12T09:55:45.613 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:45 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-12T09:55:45.614 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:45 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:45 smithi042 ceph-mon[98894]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-12-12T09:55:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:45 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-12T09:55:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:45 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:45 smithi111 ceph-mon[102565]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2023-12-12T09:55:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:45 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-12T09:55:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:45 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:46.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:46 smithi173 ceph-mon[103308]: Deploying daemon osd.11 on smithi173 2023-12-12T09:55:46.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:46 smithi042 ceph-mon[98894]: Deploying daemon osd.11 on smithi173 2023-12-12T09:55:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:46 smithi111 ceph-mon[102565]: Deploying daemon osd.11 on smithi173 2023-12-12T09:55:47.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:47 smithi173 ceph-mon[103308]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-12-12T09:55:47.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:47 smithi042 ceph-mon[98894]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-12-12T09:55:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:47 smithi111 ceph-mon[102565]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 68 KiB/s, 0 objects/s recovering 2023-12-12T09:55:49.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:49 smithi173 ceph-mon[103308]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-12T09:55:49.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:49 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:49.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:49 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:49.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:49 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:49.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:49 smithi042 ceph-mon[98894]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-12T09:55:49.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:49 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:49.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:49 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:49.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:49 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:49.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:49 smithi111 ceph-mon[102565]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-12T09:55:49.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:49 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:49.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:49 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:49.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:49 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:51.621 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:51 smithi173 ceph-mon[103308]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-12-12T09:55:51.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:51 smithi042 ceph-mon[98894]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-12-12T09:55:51.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:51 smithi111 ceph-mon[102565]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 50 KiB/s, 0 objects/s recovering 2023-12-12T09:55:51.970 INFO:teuthology.orchestra.run.smithi173.stdout:Created osd(s) 11 on host 'smithi173' 2023-12-12T09:55:51.971 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.968+0000 7fd8037fe700 1 -- 172.21.15.173:0/641019429 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fd814074f20 con 0x7fd7fc0a1500 2023-12-12T09:55:51.972 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7fc0a1500 msgr2=0x7fd7fc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:51.972 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7fc0a1500 0x7fd7fc0a39c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd8040038c0 tx=0x7fd804005fb0 comp rx=0 tx=0).stop 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814102820 msgr2=0x7fd814199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814102820 0x7fd814199e00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd80800b560 tx=0x7fd80800b870 comp rx=0 tx=0).stop 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 shutdown_connections 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7fc0a1500 0x7fd7fc0a39c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8141031c0 0x7fd8141a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd814101620 0x7fd8141998c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 --2- 172.21.15.173:0/641019429 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd814102820 0x7fd814199e00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.970+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 >> 172.21.15.173:0/641019429 conn(0x7fd8140fcbc0 msgr2=0x7fd814073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.971+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 shutdown_connections 2023-12-12T09:55:51.973 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:55:51.971+0000 7fd818860700 1 -- 172.21.15.173:0/641019429 wait complete. 2023-12-12T09:55:52.355 DEBUG:teuthology.orchestra.run.smithi173:osd.11> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.11.service 2023-12-12T09:55:52.357 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-12-12T09:55:52.357 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd stat -f json 2023-12-12T09:55:52.385 INFO:journalctl@ceph.osd.11.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:55:52.975 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:55:53.230 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:52 smithi042 ceph-mon[98894]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-12T09:55:53.231 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:52 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.231 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:52 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.231 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:52 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.231 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:52 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.232 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:52 smithi173 ceph-mon[103308]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-12T09:55:53.232 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:52 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.232 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:52 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.233 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:52 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.233 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:52 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:52 smithi111 ceph-mon[102565]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2023-12-12T09:55:53.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:52 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:52 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:52 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:52 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:53.573 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 09:55:53 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T09:55:53.236+0000 7fc1c6677640 -1 osd.11 0 log_to_monitors true 2023-12-12T09:55:54.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:53 smithi173 ceph-mon[103308]: from='osd.11 [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-12T09:55:54.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:53 smithi173 ceph-mon[103308]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-12T09:55:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 -- 172.21.15.42:0/4042940246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d7c1020b0 msgr2=0x7f5d7c108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 --2- 172.21.15.42:0/4042940246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d7c1020b0 0x7f5d7c108960 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5d7400b0d0 tx=0x7f5d7400b3e0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 -- 172.21.15.42:0/4042940246 shutdown_connections 2023-12-12T09:55:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 --2- 172.21.15.42:0/4042940246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d7c1020b0 0x7f5d7c108960 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 --2- 172.21.15.42:0/4042940246 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d7c100510 0x7f5d7c100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 --2- 172.21.15.42:0/4042940246 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d7c101710 0x7f5d7c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 -- 172.21.15.42:0/4042940246 >> 172.21.15.42:0/4042940246 conn(0x7f5d7c0fbad0 msgr2=0x7f5d7c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 -- 172.21.15.42:0/4042940246 shutdown_connections 2023-12-12T09:55:54.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 -- 172.21.15.42:0/4042940246 wait complete. 2023-12-12T09:55:54.181 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 Processor -- start 2023-12-12T09:55:54.181 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.177+0000 7f5d82f12700 1 -- start start 2023-12-12T09:55:54.181 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d82f12700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d7c100510 0x7f5d7c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:54.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d82f12700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d7c101710 0x7f5d7c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:54.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d82f12700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d7c1020b0 0x7f5d7c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:54.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d82f12700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c19f130 con 0x7f5d7c101710 2023-12-12T09:55:54.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d82f12700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c19f270 con 0x7f5d7c1020b0 2023-12-12T09:55:54.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d82f12700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c19f400 con 0x7f5d7c100510 2023-12-12T09:55:54.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d80cae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d7c100510 0x7f5d7c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:54.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d7bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d7c101710 0x7f5d7c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:54.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d7bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d7c101710 0x7f5d7c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33014/0 (socket says 172.21.15.42:33014) 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d7bfff700 1 -- 172.21.15.42:0/2479323925 learned_addr learned my addr 172.21.15.42:0/2479323925 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d814af700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d7c1020b0 0x7f5d7c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d80cae700 1 -- 172.21.15.42:0/2479323925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d7c1020b0 msgr2=0x7f5d7c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d80cae700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d7c1020b0 0x7f5d7c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d80cae700 1 -- 172.21.15.42:0/2479323925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d7c101710 msgr2=0x7f5d7c197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d80cae700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d7c101710 0x7f5d7c197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d80cae700 1 -- 172.21.15.42:0/2479323925 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d74009d00 con 0x7f5d7c100510 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d80cae700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d7c100510 0x7f5d7c197640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000e9f0 tx=0x7f5d7000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d79ffb700 1 -- 172.21.15.42:0/2479323925 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7000eee0 con 0x7f5d7c100510 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c19f6e0 con 0x7f5d7c100510 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d79ffb700 1 -- 172.21.15.42:0/2479323925 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d700044e0 con 0x7f5d7c100510 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.178+0000 7f5d79ffb700 1 -- 172.21.15.42:0/2479323925 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70003c70 con 0x7f5d7c100510 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.179+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c19fc30 con 0x7f5d7c100510 2023-12-12T09:55:54.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.180+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c04f290 con 0x7f5d7c100510 2023-12-12T09:55:54.184 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.181+0000 7f5d79ffb700 1 -- 172.21.15.42:0/2479323925 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f5d7000ccd0 con 0x7f5d7c100510 2023-12-12T09:55:54.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.181+0000 7f5d79ffb700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d6c0a1480 0x7f5d6c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:54.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.182+0000 7f5d7bfff700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d6c0a1480 0x7f5d6c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:54.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.182+0000 7f5d79ffb700 1 -- 172.21.15.42:0/2479323925 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f5d70014070 con 0x7f5d7c100510 2023-12-12T09:55:54.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.182+0000 7f5d7bfff700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d6c0a1480 0x7f5d6c0a3940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005950 tx=0x7f5d6800a400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:54.187 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.185+0000 7f5d79ffb700 1 -- 172.21.15.42:0/2479323925 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5d70096780 con 0x7f5d7c100510 2023-12-12T09:55:54.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:53 smithi111 ceph-mon[102565]: from='osd.11 [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-12T09:55:54.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:53 smithi111 ceph-mon[102565]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-12T09:55:54.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.390+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f5d7c10bd20 con 0x7f5d7c100510 2023-12-12T09:55:54.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.390+0000 7f5d79ffb700 1 -- 172.21.15.42:0/2479323925 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f5d70099da0 con 0x7f5d7c100510 2023-12-12T09:55:54.392 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:55:54.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d6c0a1480 msgr2=0x7f5d6c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d6c0a1480 0x7f5d6c0a3940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d68005950 tx=0x7f5d6800a400 comp rx=0 tx=0).stop 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d7c100510 msgr2=0x7f5d7c197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d7c100510 0x7f5d7c197640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000e9f0 tx=0x7f5d7000ed00 comp rx=0 tx=0).stop 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 shutdown_connections 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d6c0a1480 0x7f5d6c0a3940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d7c1020b0 0x7f5d7c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d7c100510 0x7f5d7c197640 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 --2- 172.21.15.42:0/2479323925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d7c101710 0x7f5d7c197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 >> 172.21.15.42:0/2479323925 conn(0x7f5d7c0fbad0 msgr2=0x7f5d7c0fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.392+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 shutdown_connections 2023-12-12T09:55:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:54.393+0000 7f5d82f12700 1 -- 172.21.15.42:0/2479323925 wait complete. 2023-12-12T09:55:54.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:53 smithi042 ceph-mon[98894]: from='osd.11 [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-12T09:55:54.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:53 smithi042 ceph-mon[98894]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-12T09:55:54.826 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":76,"num_osds":12,"num_up_osds":11,"osd_up_since":1702374933,"num_in_osds":12,"osd_in_since":1702374938,"num_remapped_pgs":0} 2023-12-12T09:55:55.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:54 smithi042 ceph-mon[98894]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-12T09:55:55.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:54 smithi042 ceph-mon[98894]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-12T09:55:55.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:54 smithi042 ceph-mon[98894]: osdmap e76: 12 total, 11 up, 12 in 2023-12-12T09:55:55.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:54 smithi042 ceph-mon[98894]: from='osd.11 [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:55:55.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:54 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:55.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:54 smithi042 ceph-mon[98894]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:55:55.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:54 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2479323925' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-12T09:55:55.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:54 smithi111 ceph-mon[102565]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-12T09:55:55.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:54 smithi111 ceph-mon[102565]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-12T09:55:55.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:54 smithi111 ceph-mon[102565]: osdmap e76: 12 total, 11 up, 12 in 2023-12-12T09:55:55.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:54 smithi111 ceph-mon[102565]: from='osd.11 [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:55:55.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:54 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:55.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:54 smithi111 ceph-mon[102565]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:55:55.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:54 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2479323925' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-12T09:55:55.323 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 09:55:54 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T09:55:54.976+0000 7fc1bbcee700 -1 osd.11 0 waiting for initial osdmap 2023-12-12T09:55:55.323 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 09:55:54 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T09:55:54.993+0000 7fc1b3ae0700 -1 osd.11 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-12T09:55:55.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:54 smithi173 ceph-mon[103308]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-12T09:55:55.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:54 smithi173 ceph-mon[103308]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-12T09:55:55.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:54 smithi173 ceph-mon[103308]: osdmap e76: 12 total, 11 up, 12 in 2023-12-12T09:55:55.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:54 smithi173 ceph-mon[103308]: from='osd.11 [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:55:55.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:54 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:55.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:54 smithi173 ceph-mon[103308]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]: dispatch 2023-12-12T09:55:55.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:54 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2479323925' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-12T09:55:55.828 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd stat -f json 2023-12-12T09:55:56.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:55:56.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: osdmap e77: 12 total, 11 up, 12 in 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: Detected new or changed devices on smithi173 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi173 to 4292M 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:56.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:56.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:56.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:56.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:56.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:55 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:55:56.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: osdmap e77: 12 total, 11 up, 12 in 2023-12-12T09:55:56.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:56.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:56.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: Detected new or changed devices on smithi173 2023-12-12T09:55:56.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi173 to 4292M 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:56.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:55 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi173", "root=default"]}]': finished 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: osdmap e77: 12 total, 11 up, 12 in 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: Detected new or changed devices on smithi173 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:55:56.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi173 to 4292M 2023-12-12T09:55:56.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:56.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:56.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:55:56.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:55:56.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:55:56.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:55 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:55:56.436 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:55:57.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:56 smithi042 ceph-mon[98894]: purged_snaps scrub starts 2023-12-12T09:55:57.239 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:56 smithi042 ceph-mon[98894]: purged_snaps scrub ok 2023-12-12T09:55:57.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:56 smithi042 ceph-mon[98894]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-12T09:55:57.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:56 smithi042 ceph-mon[98894]: osd.11 [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] boot 2023-12-12T09:55:57.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:56 smithi042 ceph-mon[98894]: osdmap e78: 12 total, 12 up, 12 in 2023-12-12T09:55:57.240 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:56 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:57.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:56 smithi111 ceph-mon[102565]: purged_snaps scrub starts 2023-12-12T09:55:57.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:56 smithi111 ceph-mon[102565]: purged_snaps scrub ok 2023-12-12T09:55:57.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:56 smithi111 ceph-mon[102565]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-12T09:55:57.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:56 smithi111 ceph-mon[102565]: osd.11 [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] boot 2023-12-12T09:55:57.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:56 smithi111 ceph-mon[102565]: osdmap e78: 12 total, 12 up, 12 in 2023-12-12T09:55:57.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:56 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:57.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:56 smithi173 ceph-mon[103308]: purged_snaps scrub starts 2023-12-12T09:55:57.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:56 smithi173 ceph-mon[103308]: purged_snaps scrub ok 2023-12-12T09:55:57.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:56 smithi173 ceph-mon[103308]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 983 GiB / 983 GiB avail 2023-12-12T09:55:57.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:56 smithi173 ceph-mon[103308]: osd.11 [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] boot 2023-12-12T09:55:57.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:56 smithi173 ceph-mon[103308]: osdmap e78: 12 total, 12 up, 12 in 2023-12-12T09:55:57.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:56 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.608+0000 7f19bc496700 1 -- 172.21.15.42:0/3394416957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 msgr2=0x7f19b4100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.608+0000 7f19bc496700 1 --2- 172.21.15.42:0/3394416957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 0x7f19b4100930 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f19a40099c0 tx=0x7f19a4009cd0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.608+0000 7f19bc496700 1 -- 172.21.15.42:0/3394416957 shutdown_connections 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.608+0000 7f19bc496700 1 --2- 172.21.15.42:0/3394416957 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19b41020b0 0x7f19b4108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.608+0000 7f19bc496700 1 --2- 172.21.15.42:0/3394416957 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 0x7f19b4100930 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.608+0000 7f19bc496700 1 --2- 172.21.15.42:0/3394416957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b4101710 0x7f19b4101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.608+0000 7f19bc496700 1 -- 172.21.15.42:0/3394416957 >> 172.21.15.42:0/3394416957 conn(0x7f19b40fbad0 msgr2=0x7f19b40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.608+0000 7f19bc496700 1 -- 172.21.15.42:0/3394416957 shutdown_connections 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.608+0000 7f19bc496700 1 -- 172.21.15.42:0/3394416957 wait complete. 2023-12-12T09:55:57.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19bc496700 1 Processor -- start 2023-12-12T09:55:57.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19bc496700 1 -- start start 2023-12-12T09:55:57.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19bc496700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 0x7f19b406a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:57.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19bc496700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b4101710 0x7f19b419c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:57.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19bc496700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19b41020b0 0x7f19b419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:57.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19bc496700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b419f0f0 con 0x7f19b4100510 2023-12-12T09:55:57.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19bc496700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b419f260 con 0x7f19b41020b0 2023-12-12T09:55:57.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19bc496700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b419f420 con 0x7f19b4101710 2023-12-12T09:55:57.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19ba232700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 0x7f19b406a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:57.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19ba232700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 0x7f19b406a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33040/0 (socket says 172.21.15.42:33040) 2023-12-12T09:55:57.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19ba232700 1 -- 172.21.15.42:0/2414178398 learned_addr learned my addr 172.21.15.42:0/2414178398 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:55:57.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19baa33700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19b41020b0 0x7f19b419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.609+0000 7f19b9a31700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b4101710 0x7f19b419c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19ba232700 1 -- 172.21.15.42:0/2414178398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b4101710 msgr2=0x7f19b419c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19ba232700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b4101710 0x7f19b419c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19ba232700 1 -- 172.21.15.42:0/2414178398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19b41020b0 msgr2=0x7f19b419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19ba232700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19b41020b0 0x7f19b419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19ba232700 1 -- 172.21.15.42:0/2414178398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a40095f0 con 0x7f19b4100510 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19b9a31700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b4101710 0x7f19b419c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19ba232700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 0x7f19b406a370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19a4003720 tx=0x7f19a4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19ab7fe700 1 -- 172.21.15.42:0/2414178398 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a401d070 con 0x7f19b4100510 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b419f6a0 con 0x7f19b4100510 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19ab7fe700 1 -- 172.21.15.42:0/2414178398 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19a4004120 con 0x7f19b4100510 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19ab7fe700 1 -- 172.21.15.42:0/2414178398 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4022e70 con 0x7f19b4100510 2023-12-12T09:55:57.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.610+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b419fb10 con 0x7f19b4100510 2023-12-12T09:55:57.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.611+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b4107a40 con 0x7f19b4100510 2023-12-12T09:55:57.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.613+0000 7f19ab7fe700 1 -- 172.21.15.42:0/2414178398 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f19a4003c60 con 0x7f19b4100510 2023-12-12T09:55:57.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.613+0000 7f19ab7fe700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f19a00a1480 0x7f19a00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:55:57.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.613+0000 7f19b9a31700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f19a00a1480 0x7f19a00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:55:57.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.613+0000 7f19ab7fe700 1 -- 172.21.15.42:0/2414178398 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f19a40cdf60 con 0x7f19b4100510 2023-12-12T09:55:57.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.614+0000 7f19b9a31700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f19a00a1480 0x7f19a00a3940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19b00099b0 tx=0x7f19b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:55:57.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.617+0000 7f19ab7fe700 1 -- 172.21.15.42:0/2414178398 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f19a4097af0 con 0x7f19b4100510 2023-12-12T09:55:57.819 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.817+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f19b410be30 con 0x7f19b4100510 2023-12-12T09:55:57.819 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.817+0000 7f19ab7fe700 1 -- 172.21.15.42:0/2414178398 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7f19a409b110 con 0x7f19b4100510 2023-12-12T09:55:57.819 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:55:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f19a00a1480 msgr2=0x7f19a00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:57.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f19a00a1480 0x7f19a00a3940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f19b00099b0 tx=0x7f19b0008040 comp rx=0 tx=0).stop 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 msgr2=0x7f19b406a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 0x7f19b406a370 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f19a4003720 tx=0x7f19a4005740 comp rx=0 tx=0).stop 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 shutdown_connections 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f19a00a1480 0x7f19a00a3940 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19b41020b0 0x7f19b419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19b4100510 0x7f19b406a370 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 --2- 172.21.15.42:0/2414178398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b4101710 0x7f19b419c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 >> 172.21.15.42:0/2414178398 conn(0x7f19b40fbad0 msgr2=0x7f19b4104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 shutdown_connections 2023-12-12T09:55:57.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:55:57.819+0000 7f19bc496700 1 -- 172.21.15.42:0/2414178398 wait complete. 2023-12-12T09:55:58.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:57 smithi042 ceph-mon[98894]: osdmap e79: 12 total, 12 up, 12 in 2023-12-12T09:55:58.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:57 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2414178398' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-12T09:55:58.193 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"num_osds":12,"num_up_osds":12,"osd_up_since":1702374955,"num_in_osds":12,"osd_in_since":1702374938,"num_remapped_pgs":0} 2023-12-12T09:55:58.193 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd dump --format=json 2023-12-12T09:55:58.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:57 smithi111 ceph-mon[102565]: osdmap e79: 12 total, 12 up, 12 in 2023-12-12T09:55:58.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:57 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2414178398' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-12T09:55:58.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:57 smithi173 ceph-mon[103308]: osdmap e79: 12 total, 12 up, 12 in 2023-12-12T09:55:58.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:57 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2414178398' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-12T09:55:58.797 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:55:59.355 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:55:59 smithi042 ceph-mon[98894]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:55:59.572 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:55:59 smithi173 ceph-mon[103308]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:55:59.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:55:59 smithi111 ceph-mon[102565]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.011+0000 7f5e6cc31700 1 -- 172.21.15.42:0/4011162805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101a40 msgr2=0x7f5e6810d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.011+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/4011162805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101a40 0x7f5e6810d3a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000b0a0 tx=0x7f5e6000b3b0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.011+0000 7f5e6cc31700 1 -- 172.21.15.42:0/4011162805 shutdown_connections 2023-12-12T09:56:00.013 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.011+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/4011162805 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101a40 0x7f5e6810d3a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.011+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/4011162805 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e68105500 0x7f5e681058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.011+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/4011162805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68101080 0x7f5e68101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.011+0000 7f5e6cc31700 1 -- 172.21.15.42:0/4011162805 >> 172.21.15.42:0/4011162805 conn(0x7f5e680757f0 msgr2=0x7f5e68075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:00.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.011+0000 7f5e6cc31700 1 -- 172.21.15.42:0/4011162805 shutdown_connections 2023-12-12T09:56:00.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.011+0000 7f5e6cc31700 1 -- 172.21.15.42:0/4011162805 wait complete. 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6cc31700 1 Processor -- start 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6cc31700 1 -- start start 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6cc31700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101080 0x7f5e6819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6cc31700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68101a40 0x7f5e6819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6cc31700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e68105500 0x7f5e681979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6cc31700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e68198180 con 0x7f5e68101080 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6cc31700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681982f0 con 0x7f5e68101a40 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6cc31700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681984b0 con 0x7f5e68105500 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101080 0x7f5e6819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101080 0x7f5e6819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33052/0 (socket says 172.21.15.42:33052) 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e6659c700 1 -- 172.21.15.42:0/3492532012 learned_addr learned my addr 172.21.15.42:0/3492532012 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:00.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e66d9d700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e68105500 0x7f5e681979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.012+0000 7f5e5ffff700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68101a40 0x7f5e6819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e6659c700 1 -- 172.21.15.42:0/3492532012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e68105500 msgr2=0x7f5e681979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e6659c700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e68105500 0x7f5e681979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e6659c700 1 -- 172.21.15.42:0/3492532012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68101a40 msgr2=0x7f5e6819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e6659c700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68101a40 0x7f5e6819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e6659c700 1 -- 172.21.15.42:0/3492532012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e60009d00 con 0x7f5e68101080 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e6659c700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101080 0x7f5e6819d920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c920 tx=0x7f5e5800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e5f7fe700 1 -- 172.21.15.42:0/3492532012 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58007b00 con 0x7f5e68101080 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e5f7fe700 1 -- 172.21.15.42:0/3492532012 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e5800ce70 con 0x7f5e68101080 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e68198790 con 0x7f5e68101080 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e5f7fe700 1 -- 172.21.15.42:0/3492532012 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580054e0 con 0x7f5e68101080 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.013+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e681a48a0 con 0x7f5e68101080 2023-12-12T09:56:00.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.014+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e68109b00 con 0x7f5e68101080 2023-12-12T09:56:00.018 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.015+0000 7f5e5f7fe700 1 -- 172.21.15.42:0/3492532012 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f5e58004750 con 0x7f5e68101080 2023-12-12T09:56:00.018 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.016+0000 7f5e5f7fe700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e540a1480 0x7f5e540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:00.018 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.016+0000 7f5e5f7fe700 1 -- 172.21.15.42:0/3492532012 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5e580cc810 con 0x7f5e68101080 2023-12-12T09:56:00.018 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.016+0000 7f5e5ffff700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e540a1480 0x7f5e540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:00.018 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.016+0000 7f5e5ffff700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e540a1480 0x7f5e540a3940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5e50006fd0 tx=0x7f5e50008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:00.021 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.020+0000 7f5e5f7fe700 1 -- 172.21.15.42:0/3492532012 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5e58096310 con 0x7f5e68101080 2023-12-12T09:56:00.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.218+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e68068af0 con 0x7f5e68101080 2023-12-12T09:56:00.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.219+0000 7f5e5f7fe700 1 -- 172.21.15.42:0/3492532012 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18411 (secure 0 0 0) 0x7f5e58099930 con 0x7f5e68101080 2023-12-12T09:56:00.222 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:00.222 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:55:56.982365+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e540a1480 msgr2=0x7f5e540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e540a1480 0x7f5e540a3940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f5e50006fd0 tx=0x7f5e50008040 comp rx=0 tx=0).stop 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101080 msgr2=0x7f5e6819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101080 0x7f5e6819d920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c920 tx=0x7f5e5800cc30 comp rx=0 tx=0).stop 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 shutdown_connections 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e540a1480 0x7f5e540a3940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e68105500 0x7f5e681979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e68101080 0x7f5e6819d920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 --2- 172.21.15.42:0/3492532012 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e68101a40 0x7f5e6819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 >> 172.21.15.42:0/3492532012 conn(0x7f5e680757f0 msgr2=0x7f5e68100130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:00.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 shutdown_connections 2023-12-12T09:56:00.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:00.222+0000 7f5e6cc31700 1 -- 172.21.15.42:0/3492532012 wait complete. 2023-12-12T09:56:00.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:00 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3492532012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:00.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:00 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3492532012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:00.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:00 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3492532012' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:00.763 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:56:00.763 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-12-12T09:56:01.364 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:56:01.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:01 smithi173 ceph-mon[103308]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:01 smithi042 ceph-mon[98894]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:01 smithi111 ceph-mon[102565]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.887+0000 7f03eb1f2700 1 -- 172.21.15.42:0/918210742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e406a580 msgr2=0x7f03e410dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.887+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/918210742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e406a580 0x7f03e410dba0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f03d00098f0 tx=0x7f03d0009c00 comp rx=0 tx=0).stop 2023-12-12T09:56:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.887+0000 7f03eb1f2700 1 -- 172.21.15.42:0/918210742 shutdown_connections 2023-12-12T09:56:02.889 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.887+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/918210742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e4069c60 0x7f03e406a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:02.890 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.887+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/918210742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e410e0e0 0x7f03e41104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:02.890 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.887+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/918210742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e406a580 0x7f03e410dba0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:02.890 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.887+0000 7f03eb1f2700 1 -- 172.21.15.42:0/918210742 >> 172.21.15.42:0/918210742 conn(0x7f03e40765b0 msgr2=0x7f03e40769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:02.890 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.887+0000 7f03eb1f2700 1 -- 172.21.15.42:0/918210742 shutdown_connections 2023-12-12T09:56:02.890 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.887+0000 7f03eb1f2700 1 -- 172.21.15.42:0/918210742 wait complete. 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03eb1f2700 1 Processor -- start 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03eb1f2700 1 -- start start 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03eb1f2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e4069c60 0x7f03e419f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03eb1f2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e406a580 0x7f03e419ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03eb1f2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e410e0e0 0x7f03e4199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03eb1f2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e419a240 con 0x7f03e4069c60 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03eb1f2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e419a3b0 con 0x7f03e410e0e0 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03eb1f2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e419a570 con 0x7f03e406a580 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e8f8e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e4069c60 0x7f03e419f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:02.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e8f8e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e4069c60 0x7f03e419f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33062/0 (socket says 172.21.15.42:33062) 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e8f8e700 1 -- 172.21.15.42:0/3857407209 learned_addr learned my addr 172.21.15.42:0/3857407209 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e3fff700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e406a580 0x7f03e419ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e978f700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e410e0e0 0x7f03e4199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e8f8e700 1 -- 172.21.15.42:0/3857407209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e406a580 msgr2=0x7f03e419ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e8f8e700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e406a580 0x7f03e419ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e8f8e700 1 -- 172.21.15.42:0/3857407209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e410e0e0 msgr2=0x7f03e4199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e8f8e700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e410e0e0 0x7f03e4199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e8f8e700 1 -- 172.21.15.42:0/3857407209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d80095f0 con 0x7f03e4069c60 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e3fff700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e406a580 0x7f03e419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.888+0000 7f03e978f700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e410e0e0 0x7f03e4199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.889+0000 7f03e8f8e700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e4069c60 0x7f03e419f9e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03d800ec10 tx=0x7f03d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.889+0000 7f03e1ffb700 1 -- 172.21.15.42:0/3857407209 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8009a10 con 0x7f03e4069c60 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.889+0000 7f03e1ffb700 1 -- 172.21.15.42:0/3857407209 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f03d80044e0 con 0x7f03e4069c60 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.889+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d0009520 con 0x7f03e4069c60 2023-12-12T09:56:02.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.889+0000 7f03e1ffb700 1 -- 172.21.15.42:0/3857407209 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8010490 con 0x7f03e4069c60 2023-12-12T09:56:02.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.889+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e41a6960 con 0x7f03e4069c60 2023-12-12T09:56:02.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.890+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e4109300 con 0x7f03e4069c60 2023-12-12T09:56:02.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.891+0000 7f03e1ffb700 1 -- 172.21.15.42:0/3857407209 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f03d8003680 con 0x7f03e4069c60 2023-12-12T09:56:02.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.892+0000 7f03e1ffb700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f03d40a1500 0x7f03d40a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:02.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.892+0000 7f03e3fff700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f03d40a1500 0x7f03d40a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:02.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.892+0000 7f03e1ffb700 1 -- 172.21.15.42:0/3857407209 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f03d8014070 con 0x7f03e4069c60 2023-12-12T09:56:02.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.892+0000 7f03e3fff700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f03d40a1500 0x7f03d40a39c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009fd0 tx=0x7f03d0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:02.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:02.896+0000 7f03e1ffb700 1 -- 172.21.15.42:0/3857407209 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f03d8096960 con 0x7f03e4069c60 2023-12-12T09:56:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.095+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03e419af70 con 0x7f03e4069c60 2023-12-12T09:56:03.098 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.096+0000 7f03e1ffb700 1 -- 172.21.15.42:0/3857407209 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f03d8099f80 con 0x7f03e4069c60 2023-12-12T09:56:03.098 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T09:56:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f03d40a1500 msgr2=0x7f03d40a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f03d40a1500 0x7f03d40a39c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f03d0009fd0 tx=0x7f03d0019040 comp rx=0 tx=0).stop 2023-12-12T09:56:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e4069c60 msgr2=0x7f03e419f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:03.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e4069c60 0x7f03e419f9e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f03d800ec10 tx=0x7f03d800c2d0 comp rx=0 tx=0).stop 2023-12-12T09:56:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 shutdown_connections 2023-12-12T09:56:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f03d40a1500 0x7f03d40a39c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03e410e0e0 0x7f03e4199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03e4069c60 0x7f03e419f9e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 --2- 172.21.15.42:0/3857407209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03e406a580 0x7f03e419ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 >> 172.21.15.42:0/3857407209 conn(0x7f03e40765b0 msgr2=0x7f03e410f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 shutdown_connections 2023-12-12T09:56:03.101 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:03.098+0000 7f03eb1f2700 1 -- 172.21.15.42:0/3857407209 wait complete. 2023-12-12T09:56:03.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:03 smithi042 ceph-mon[98894]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:03.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:03 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3857407209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:03.531 INFO:tasks.cephadm:Adding mds.a on smithi042 2023-12-12T09:56:03.531 INFO:tasks.cephadm:Adding mds.d on smithi042 2023-12-12T09:56:03.531 INFO:tasks.cephadm:Adding mds.g on smithi042 2023-12-12T09:56:03.531 INFO:tasks.cephadm:Adding mds.j on smithi042 2023-12-12T09:56:03.531 INFO:tasks.cephadm:Adding mds.b on smithi111 2023-12-12T09:56:03.531 INFO:tasks.cephadm:Adding mds.e on smithi111 2023-12-12T09:56:03.532 INFO:tasks.cephadm:Adding mds.h on smithi111 2023-12-12T09:56:03.532 INFO:tasks.cephadm:Adding mds.k on smithi111 2023-12-12T09:56:03.532 INFO:tasks.cephadm:Adding mds.c on smithi173 2023-12-12T09:56:03.532 INFO:tasks.cephadm:Adding mds.f on smithi173 2023-12-12T09:56:03.532 INFO:tasks.cephadm:Adding mds.i on smithi173 2023-12-12T09:56:03.532 INFO:tasks.cephadm:Adding mds.l on smithi173 2023-12-12T09:56:03.532 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph orch apply mds all '12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l' 2023-12-12T09:56:03.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:03 smithi173 ceph-mon[103308]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:03.565 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:03 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3857407209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:03.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:03 smithi111 ceph-mon[102565]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:03.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:03 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3857407209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:04.143 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T09:56:05.323 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.320+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2269955363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214101710 msgr2=0x7fc214101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:05.323 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.320+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2269955363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214101710 0x7fc214101b70 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc204009990 tx=0x7fc204009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.323 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.321+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2269955363 shutdown_connections 2023-12-12T09:56:05.323 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.321+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2269955363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc2141020b0 0x7fc214108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.323 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.321+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2269955363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc214100510 0x7fc214100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.323 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.321+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2269955363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214101710 0x7fc214101b70 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.323 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.321+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2269955363 >> 172.21.15.173:0/2269955363 conn(0x7fc2140fbad0 msgr2=0x7fc2140fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:05.323 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.321+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2269955363 shutdown_connections 2023-12-12T09:56:05.323 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.321+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2269955363 wait complete. 2023-12-12T09:56:05.325 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.321+0000 7fc21c6ef700 1 Processor -- start 2023-12-12T09:56:05.325 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.321+0000 7fc21c6ef700 1 -- start start 2023-12-12T09:56:05.325 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21c6ef700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214100510 0x7fc214196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21c6ef700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc214101710 0x7fc214196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21c6ef700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2141020b0 0x7fc21419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21c6ef700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21419df50 con 0x7fc2141020b0 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21c6ef700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21419e090 con 0x7fc214101710 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21c6ef700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21419e220 con 0x7fc214100510 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21a48b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214100510 0x7fc214196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21a48b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214100510 0x7fc214196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:42978/0 (socket says 172.21.15.173:42978) 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21a48b700 1 -- 172.21.15.173:0/2445385483 learned_addr learned my addr 172.21.15.173:0/2445385483 (peer_addr_for_me v2:172.21.15.173:0/0) 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21ac8c700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2141020b0 0x7fc21419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc219c8a700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc214101710 0x7fc214196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21a48b700 1 -- 172.21.15.173:0/2445385483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc214101710 msgr2=0x7fc214196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21a48b700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc214101710 0x7fc214196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21a48b700 1 -- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2141020b0 msgr2=0x7fc21419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:05.326 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21a48b700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2141020b0 0x7fc21419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21a48b700 1 -- 172.21.15.173:0/2445385483 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2040095f0 con 0x7fc214100510 2023-12-12T09:56:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc219c8a700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc214101710 0x7fc214196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:56:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21a48b700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214100510 0x7fc214196410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc210007960 tx=0x7fc210007d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc20b7fe700 1 -- 172.21.15.173:0/2445385483 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210012070 con 0x7fc214100510 2023-12-12T09:56:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc20b7fe700 1 -- 172.21.15.173:0/2445385483 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc210009bf0 con 0x7fc214100510 2023-12-12T09:56:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc214068fd0 con 0x7fc214100510 2023-12-12T09:56:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc20b7fe700 1 -- 172.21.15.173:0/2445385483 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210018ca0 con 0x7fc214100510 2023-12-12T09:56:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.322+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2140694d0 con 0x7fc214100510 2023-12-12T09:56:05.327 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.323+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21410c660 con 0x7fc214100510 2023-12-12T09:56:05.328 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.324+0000 7fc20b7fe700 1 -- 172.21.15.173:0/2445385483 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fc210016070 con 0x7fc214100510 2023-12-12T09:56:05.328 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.325+0000 7fc20b7fe700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc2000a1480 0x7fc2000a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:05.328 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.325+0000 7fc20b7fe700 1 -- 172.21.15.173:0/2445385483 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc2100cd970 con 0x7fc214100510 2023-12-12T09:56:05.328 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.325+0000 7fc219c8a700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc2000a1480 0x7fc2000a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:05.328 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.326+0000 7fc219c8a700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc2000a1480 0x7fc2000a3940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc2040038c0 tx=0x7fc204005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:05.330 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.328+0000 7fc20b7fe700 1 -- 172.21.15.173:0/2445385483 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc210097470 con 0x7fc214100510 2023-12-12T09:56:05.529 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.527+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}) v1 -- 0x7fc214107540 con 0x7fc2000a1480 2023-12-12T09:56:05.538 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.536+0000 7fc20b7fe700 1 -- 172.21.15.173:0/2445385483 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fc214107540 con 0x7fc2000a1480 2023-12-12T09:56:05.538 INFO:teuthology.orchestra.run.smithi173.stdout:Scheduled mds.all update... 2023-12-12T09:56:05.540 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc2000a1480 msgr2=0x7fc2000a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:05.540 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc2000a1480 0x7fc2000a3940 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc2040038c0 tx=0x7fc204005fb0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.540 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214100510 msgr2=0x7fc214196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:05.540 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214100510 0x7fc214196410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc210007960 tx=0x7fc210007d20 comp rx=0 tx=0).stop 2023-12-12T09:56:05.541 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 shutdown_connections 2023-12-12T09:56:05.541 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc2000a1480 0x7fc2000a3940 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.541 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc2141020b0 0x7fc21419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.541 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc214100510 0x7fc214196410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.541 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 --2- 172.21.15.173:0/2445385483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc214101710 0x7fc214196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:05.541 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 >> 172.21.15.173:0/2445385483 conn(0x7fc2140fbad0 msgr2=0x7fc214104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:05.541 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 shutdown_connections 2023-12-12T09:56:05.541 INFO:teuthology.orchestra.run.smithi173.stderr:2023-12-12T09:56:05.538+0000 7fc21c6ef700 1 -- 172.21.15.173:0/2445385483 wait complete. 2023-12-12T09:56:05.561 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:05 smithi173 ceph-mon[103308]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:05.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:05 smithi042 ceph-mon[98894]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:05 smithi111 ceph-mon[102565]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:05.891 DEBUG:teuthology.orchestra.run.smithi042:mds.a> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.a.service 2023-12-12T09:56:05.894 DEBUG:teuthology.orchestra.run.smithi042:mds.d> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.d.service 2023-12-12T09:56:05.932 INFO:journalctl@ceph.mds.a.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:56:05.933 DEBUG:teuthology.orchestra.run.smithi042:mds.g> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.g.service 2023-12-12T09:56:05.963 INFO:journalctl@ceph.mds.d.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:56:05.964 DEBUG:teuthology.orchestra.run.smithi042:mds.j> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.j.service 2023-12-12T09:56:05.992 INFO:journalctl@ceph.mds.g.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:56:05.994 DEBUG:teuthology.orchestra.run.smithi111:mds.b> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.b.service 2023-12-12T09:56:05.996 DEBUG:teuthology.orchestra.run.smithi111:mds.e> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.e.service 2023-12-12T09:56:06.021 INFO:journalctl@ceph.mds.j.smithi042.stdout:-- Logs begin at Tue 2023-12-12 09:33:40 UTC. -- 2023-12-12T09:56:06.034 INFO:journalctl@ceph.mds.b.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:56:06.035 DEBUG:teuthology.orchestra.run.smithi111:mds.h> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.h.service 2023-12-12T09:56:06.065 INFO:journalctl@ceph.mds.e.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:56:06.066 DEBUG:teuthology.orchestra.run.smithi111:mds.k> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.k.service 2023-12-12T09:56:06.093 INFO:journalctl@ceph.mds.h.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:56:06.094 DEBUG:teuthology.orchestra.run.smithi173:mds.c> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.c.service 2023-12-12T09:56:06.097 DEBUG:teuthology.orchestra.run.smithi173:mds.f> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.f.service 2023-12-12T09:56:06.120 INFO:journalctl@ceph.mds.k.smithi111.stdout:-- Logs begin at Tue 2023-12-12 09:34:28 UTC. -- 2023-12-12T09:56:06.124 INFO:journalctl@ceph.mds.c.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:56:06.126 DEBUG:teuthology.orchestra.run.smithi173:mds.i> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.i.service 2023-12-12T09:56:06.151 INFO:journalctl@ceph.mds.f.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:56:06.152 DEBUG:teuthology.orchestra.run.smithi173:mds.l> sudo journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.l.service 2023-12-12T09:56:06.180 INFO:journalctl@ceph.mds.i.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:56:06.181 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-12-12T09:56:06.181 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:56:06.208 INFO:journalctl@ceph.mds.l.smithi173.stdout:-- Logs begin at Tue 2023-12-12 09:34:00 UTC. -- 2023-12-12T09:56:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 -- 172.21.15.42:0/1708824583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 msgr2=0x7fc3301087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 --2- 172.21.15.42:0/1708824583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 0x7fc3301087f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc32800b0d0 tx=0x7fc32800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 -- 172.21.15.42:0/1708824583 shutdown_connections 2023-12-12T09:56:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 --2- 172.21.15.42:0/1708824583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3301003b0 0x7fc3301007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 --2- 172.21.15.42:0/1708824583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 0x7fc3301087f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 --2- 172.21.15.42:0/1708824583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3301015a0 0x7fc330101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 -- 172.21.15.42:0/1708824583 >> 172.21.15.42:0/1708824583 conn(0x7fc3300fb920 msgr2=0x7fc3300fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 -- 172.21.15.42:0/1708824583 shutdown_connections 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 -- 172.21.15.42:0/1708824583 wait complete. 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 Processor -- start 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.327+0000 7fc336cec700 1 -- start start 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc336cec700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3301003b0 0x7fc3301974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc336cec700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3301015a0 0x7fc330197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc336cec700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 0x7fc33019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc336cec700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33019f000 con 0x7fc330101f40 2023-12-12T09:56:06.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc336cec700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33019f140 con 0x7fc3301015a0 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc336cec700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33019f2d0 con 0x7fc3301003b0 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc335289700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 0x7fc33019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc335289700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 0x7fc33019ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33078/0 (socket says 172.21.15.42:33078) 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc335289700 1 -- 172.21.15.42:0/742368634 learned_addr learned my addr 172.21.15.42:0/742368634 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc32ffff700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3301015a0 0x7fc330197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc334a88700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3301003b0 0x7fc3301974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc335289700 1 -- 172.21.15.42:0/742368634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3301003b0 msgr2=0x7fc3301974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc335289700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3301003b0 0x7fc3301974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc335289700 1 -- 172.21.15.42:0/742368634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3301015a0 msgr2=0x7fc330197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc335289700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3301015a0 0x7fc330197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc335289700 1 -- 172.21.15.42:0/742368634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc328009d00 con 0x7fc330101f40 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc32ffff700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3301015a0 0x7fc330197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:06.333 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc335289700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 0x7fc33019ea30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc328007bd0 tx=0x7fc3280094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc32dffb700 1 -- 172.21.15.42:0/742368634 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32800e040 con 0x7fc330101f40 2023-12-12T09:56:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc32dffb700 1 -- 172.21.15.42:0/742368634 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc328012dc0 con 0x7fc330101f40 2023-12-12T09:56:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33019f550 con 0x7fc330101f40 2023-12-12T09:56:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.328+0000 7fc32dffb700 1 -- 172.21.15.42:0/742368634 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32801b450 con 0x7fc330101f40 2023-12-12T09:56:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.329+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33019f9d0 con 0x7fc330101f40 2023-12-12T09:56:06.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.330+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3300679a0 con 0x7fc330101f40 2023-12-12T09:56:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.331+0000 7fc32dffb700 1 -- 172.21.15.42:0/742368634 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fc32801d610 con 0x7fc330101f40 2023-12-12T09:56:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.331+0000 7fc32dffb700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc3200a1500 0x7fc3200a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.331+0000 7fc334a88700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc3200a1500 0x7fc3200a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.331+0000 7fc32dffb700 1 -- 172.21.15.42:0/742368634 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc3280ce810 con 0x7fc330101f40 2023-12-12T09:56:06.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.332+0000 7fc334a88700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc3200a1500 0x7fc3200a39c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc324005950 tx=0x7fc3240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:06.337 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.336+0000 7fc32dffb700 1 -- 172.21.15.42:0/742368634 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc3280983a0 con 0x7fc330101f40 2023-12-12T09:56:06.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.542+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc33010bde0 con 0x7fc330101f40 2023-12-12T09:56:06.544 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.542+0000 7fc32dffb700 1 -- 172.21.15.42:0/742368634 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18411 (secure 0 0 0) 0x7fc32809b9c0 con 0x7fc330101f40 2023-12-12T09:56:06.545 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:06.545 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:55:56.982365+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:56:06.547 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc3200a1500 msgr2=0x7fc3200a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.547 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc3200a1500 0x7fc3200a39c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc324005950 tx=0x7fc3240058e0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.547 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 msgr2=0x7fc33019ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.547 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 0x7fc33019ea30 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc328007bd0 tx=0x7fc3280094f0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 shutdown_connections 2023-12-12T09:56:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc3200a1500 0x7fc3200a39c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc330101f40 0x7fc33019ea30 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3301003b0 0x7fc3301974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 --2- 172.21.15.42:0/742368634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc3301015a0 0x7fc330197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 >> 172.21.15.42:0/742368634 conn(0x7fc3300fb920 msgr2=0x7fc3300fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 shutdown_connections 2023-12-12T09:56:06.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.545+0000 7fc336cec700 1 -- 172.21.15.42:0/742368634 wait complete. 2023-12-12T09:56:06.560 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:56:06.560 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.687+0000 7f212d211700 1 -- 172.21.15.42:0/2176249518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 msgr2=0x7f21281008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.687+0000 7f212d211700 1 --2- 172.21.15.42:0/2176249518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 0x7f21281008a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f21180099c0 tx=0x7f2118009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.687+0000 7f212d211700 1 -- 172.21.15.42:0/2176249518 shutdown_connections 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.687+0000 7f212d211700 1 --2- 172.21.15.42:0/2176249518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2128102010 0x7f21281088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.687+0000 7f212d211700 1 --2- 172.21.15.42:0/2176249518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 0x7f21281008a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.687+0000 7f212d211700 1 --2- 172.21.15.42:0/2176249518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2128101670 0x7f2128101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.687+0000 7f212d211700 1 -- 172.21.15.42:0/2176249518 >> 172.21.15.42:0/2176249518 conn(0x7f21280fb9f0 msgr2=0x7f21280fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.687+0000 7f212d211700 1 -- 172.21.15.42:0/2176249518 shutdown_connections 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.687+0000 7f212d211700 1 -- 172.21.15.42:0/2176249518 wait complete. 2023-12-12T09:56:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.688+0000 7f212d211700 1 Processor -- start 2023-12-12T09:56:06.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.688+0000 7f212d211700 1 -- start start 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.688+0000 7f212d211700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 0x7f2128196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.688+0000 7f212d211700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2128101670 0x7f2128196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.688+0000 7f212d211700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2128102010 0x7f212819d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.688+0000 7f212d211700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212819df50 con 0x7f2128100480 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.688+0000 7f212d211700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212819e090 con 0x7f2128102010 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.688+0000 7f212d211700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212819e220 con 0x7f2128101670 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f2126d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 0x7f2128196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f2126d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 0x7f2128196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33088/0 (socket says 172.21.15.42:33088) 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f2126d9d700 1 -- 172.21.15.42:0/637271064 learned_addr learned my addr 172.21.15.42:0/637271064 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f212659c700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2128101670 0x7f2128196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f212759e700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2128102010 0x7f212819d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f2126d9d700 1 -- 172.21.15.42:0/637271064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2128101670 msgr2=0x7f2128196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f2126d9d700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2128101670 0x7f2128196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f2126d9d700 1 -- 172.21.15.42:0/637271064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2128102010 msgr2=0x7f212819d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f2126d9d700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2128102010 0x7f212819d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f2126d9d700 1 -- 172.21.15.42:0/637271064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21180095f0 con 0x7f2128100480 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f212759e700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2128102010 0x7f212819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f2126d9d700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 0x7f2128196420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2118006010 tx=0x7f2118005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f210ffff700 1 -- 172.21.15.42:0/637271064 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211801f070 con 0x7f2128100480 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f210ffff700 1 -- 172.21.15.42:0/637271064 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2118019d90 con 0x7f2128100480 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2128068610 con 0x7f2128100480 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f210ffff700 1 -- 172.21.15.42:0/637271064 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118024470 con 0x7f2128100480 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.689+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2128068ac0 con 0x7f2128100480 2023-12-12T09:56:06.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.690+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2128068310 con 0x7f2128100480 2023-12-12T09:56:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.691+0000 7f210ffff700 1 -- 172.21.15.42:0/637271064 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2118003a30 con 0x7f2128100480 2023-12-12T09:56:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.692+0000 7f210ffff700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f21100a1480 0x7f21100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.692+0000 7f210ffff700 1 -- 172.21.15.42:0/637271064 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f21180d0130 con 0x7f2128100480 2023-12-12T09:56:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.692+0000 7f212659c700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f21100a1480 0x7f21100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.692+0000 7f212659c700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f21100a1480 0x7f21100a3940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f211c005950 tx=0x7f211c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:06.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.695+0000 7f210ffff700 1 -- 172.21.15.42:0/637271064 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2118099cc0 con 0x7f2128100480 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:06 smithi173 ceph-mon[103308]: Deploying daemon mds.l on smithi173 2023-12-12T09:56:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:56:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2023-12-12T09:56:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:56:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:56:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:06.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:06.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:06.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:06 smithi042 ceph-mon[98894]: Deploying daemon mds.l on smithi173 2023-12-12T09:56:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.901+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f212810bce0 con 0x7f2128100480 2023-12-12T09:56:06.904 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.902+0000 7f210ffff700 1 -- 172.21.15.42:0/637271064 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f211809d2e0 con 0x7f2128100480 2023-12-12T09:56:06.904 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T09:56:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f21100a1480 msgr2=0x7f21100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f21100a1480 0x7f21100a3940 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f211c005950 tx=0x7f211c00b410 comp rx=0 tx=0).stop 2023-12-12T09:56:06.906 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 msgr2=0x7f2128196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 0x7f2128196420 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2118006010 tx=0x7f2118005740 comp rx=0 tx=0).stop 2023-12-12T09:56:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 shutdown_connections 2023-12-12T09:56:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f21100a1480 0x7f21100a3940 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2128102010 0x7f212819d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2128100480 0x7f2128196420 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 --2- 172.21.15.42:0/637271064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2128101670 0x7f2128196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 >> 172.21.15.42:0/637271064 conn(0x7f21280fb9f0 msgr2=0x7f2128104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 shutdown_connections 2023-12-12T09:56:06.907 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:06.904+0000 7f212d211700 1 -- 172.21.15.42:0/637271064 wait complete. 2023-12-12T09:56:06.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:06.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:56:06.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2023-12-12T09:56:06.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:06.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:56:06.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:06.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:56:06.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:06.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:06.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:06.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:06.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:06 smithi111 ceph-mon[102565]: Deploying daemon mds.l on smithi173 2023-12-12T09:56:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 -- 172.21.15.42:0/3305411210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68101570 msgr2=0x7f8a681019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/3305411210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68101570 0x7f8a681019d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8a580099c0 tx=0x7f8a58009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 -- 172.21.15.42:0/3305411210 shutdown_connections 2023-12-12T09:56:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/3305411210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a68101f10 0x7f8a681087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/3305411210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a68100380 0x7f8a681007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/3305411210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68101570 0x7f8a681019d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 -- 172.21.15.42:0/3305411210 >> 172.21.15.42:0/3305411210 conn(0x7f8a680fb8f0 msgr2=0x7f8a680fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:07.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 -- 172.21.15.42:0/3305411210 shutdown_connections 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 -- 172.21.15.42:0/3305411210 wait complete. 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.042+0000 7f8a6d5be700 1 Processor -- start 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a6d5be700 1 -- start start 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a6d5be700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68100380 0x7f8a68069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a6d5be700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a68101570 0x7f8a6819c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a6d5be700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a68101f10 0x7f8a6819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a6d5be700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6819ef00 con 0x7f8a68100380 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a6d5be700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6819f070 con 0x7f8a68101f10 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a6d5be700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6819f230 con 0x7f8a68101570 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a66ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68100380 0x7f8a68069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a66ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68100380 0x7f8a68069150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33102/0 (socket says 172.21.15.42:33102) 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a66ffd700 1 -- 172.21.15.42:0/1590534037 learned_addr learned my addr 172.21.15.42:0/1590534037 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a667fc700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a68101570 0x7f8a6819c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:07.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a677fe700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a68101f10 0x7f8a6819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a66ffd700 1 -- 172.21.15.42:0/1590534037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a68101570 msgr2=0x7f8a6819c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a66ffd700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a68101570 0x7f8a6819c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a66ffd700 1 -- 172.21.15.42:0/1590534037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a68101f10 msgr2=0x7f8a6819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a66ffd700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a68101f10 0x7f8a6819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a66ffd700 1 -- 172.21.15.42:0/1590534037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580095f0 con 0x7f8a68100380 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.043+0000 7f8a66ffd700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68100380 0x7f8a68069150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00e970 tx=0x7f8a5c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.044+0000 7f8a4ffff700 1 -- 172.21.15.42:0/1590534037 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c009790 con 0x7f8a68100380 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.044+0000 7f8a4ffff700 1 -- 172.21.15.42:0/1590534037 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a5c00c7e0 con 0x7f8a68100380 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.044+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6819f4b0 con 0x7f8a68100380 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.044+0000 7f8a4ffff700 1 -- 172.21.15.42:0/1590534037 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c003c70 con 0x7f8a68100380 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.044+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6819f9e0 con 0x7f8a68100380 2023-12-12T09:56:07.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.045+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68107780 con 0x7f8a68100380 2023-12-12T09:56:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.046+0000 7f8a4ffff700 1 -- 172.21.15.42:0/1590534037 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8a5c00cc50 con 0x7f8a68100380 2023-12-12T09:56:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.047+0000 7f8a4ffff700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a500a1500 0x7f8a500a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.047+0000 7f8a667fc700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a500a1500 0x7f8a500a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.047+0000 7f8a4ffff700 1 -- 172.21.15.42:0/1590534037 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a5c014070 con 0x7f8a68100380 2023-12-12T09:56:07.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.047+0000 7f8a667fc700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a500a1500 0x7f8a500a39c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a58003730 tx=0x7f8a5800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:07.052 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.050+0000 7f8a4ffff700 1 -- 172.21.15.42:0/1590534037 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8a5c0965e0 con 0x7f8a68100380 2023-12-12T09:56:07.298 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.296+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a68068230 con 0x7f8a68100380 2023-12-12T09:56:07.298 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.296+0000 7f8a4ffff700 1 -- 172.21.15.42:0/1590534037 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f8a5c099c00 con 0x7f8a68100380 2023-12-12T09:56:07.298 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:07.298 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-12-12T09:56:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a500a1500 msgr2=0x7f8a500a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a500a1500 0x7f8a500a39c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8a58003730 tx=0x7f8a5800b540 comp rx=0 tx=0).stop 2023-12-12T09:56:07.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68100380 msgr2=0x7f8a68069150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68100380 0x7f8a68069150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00e970 tx=0x7f8a5c00ec80 comp rx=0 tx=0).stop 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 shutdown_connections 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a500a1500 0x7f8a500a39c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a68101f10 0x7f8a6819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a68100380 0x7f8a68069150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 --2- 172.21.15.42:0/1590534037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a68101570 0x7f8a6819c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 >> 172.21.15.42:0/1590534037 conn(0x7f8a680fb8f0 msgr2=0x7f8a681048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 shutdown_connections 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:07.298+0000 7f8a6d5be700 1 -- 172.21.15.42:0/1590534037 wait complete. 2023-12-12T09:56:07.301 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 1 2023-12-12T09:56:07.323 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 09:56:07 smithi173 systemd[1]: Starting Ceph mds.l for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:07 smithi042 ceph-mon[98894]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:07 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/742368634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:07 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/637271064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:07 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1590534037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:07.694 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:07 smithi173 ceph-mon[103308]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:07.695 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:07 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/742368634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:07.695 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:07 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/637271064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:07.695 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:07 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1590534037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:07.695 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 09:56:07 smithi173 podman[139457]: 2023-12-12T09:56:07.695 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 09:56:07 smithi173 ceph-mds[139497]: -- 172.21.15.173:0/2236535236 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5601b34d5520 con 0x5601b34ab000 2023-12-12T09:56:07.695 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 09:56:07 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-l[139473]: starting mds.l at 2023-12-12T09:56:07.695 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 09:56:07 smithi173 bash[139457]: f79e683d410199c8880b27842ce5cef83547c6b26d29835ba771cfccc6839921 2023-12-12T09:56:07.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:07 smithi111 ceph-mon[102565]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:07.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:07 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/742368634' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:07.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:07 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/637271064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:07.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:07 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1590534037' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:08.073 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 09:56:07 smithi173 systemd[1]: Started Ceph mds.l for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:08.711 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:08 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6832/2859393185,v1:172.21.15.173:6833/2859393185] up:boot 2023-12-12T09:56:08.712 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:08 smithi111 ceph-mon[102565]: fsmap 1 up:standby 2023-12-12T09:56:08.712 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-12T09:56:08.712 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:08.712 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:08.712 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:08.713 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:08.713 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:08.713 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:08 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:08 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6832/2859393185,v1:172.21.15.173:6833/2859393185] up:boot 2023-12-12T09:56:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:08 smithi042 ceph-mon[98894]: fsmap 1 up:standby 2023-12-12T09:56:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-12T09:56:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:08.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:08 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:08 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6832/2859393185,v1:172.21.15.173:6833/2859393185] up:boot 2023-12-12T09:56:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:08 smithi173 ceph-mon[103308]: fsmap 1 up:standby 2023-12-12T09:56:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-12T09:56:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:08 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:09.247 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 09:56:09 smithi111 systemd[1]: Starting Ceph mds.k for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:09.313 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.451+0000 7fe6697fa700 1 -- 172.21.15.42:0/3595328612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664101f00 msgr2=0x7fe6641042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.451+0000 7fe6697fa700 1 --2- 172.21.15.42:0/3595328612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664101f00 0x7fe6641042f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe654009990 tx=0x7fe654009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.451+0000 7fe6697fa700 1 -- 172.21.15.42:0/3595328612 shutdown_connections 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.451+0000 7fe6697fa700 1 --2- 172.21.15.42:0/3595328612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664107160 0x7fe6641095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.451+0000 7fe6697fa700 1 --2- 172.21.15.42:0/3595328612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664101f00 0x7fe6641042f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.451+0000 7fe6697fa700 1 --2- 172.21.15.42:0/3595328612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe664104830 0x7fe664106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.451+0000 7fe6697fa700 1 -- 172.21.15.42:0/3595328612 >> 172.21.15.42:0/3595328612 conn(0x7fe6640fb930 msgr2=0x7fe6640fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.451+0000 7fe6697fa700 1 -- 172.21.15.42:0/3595328612 shutdown_connections 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.451+0000 7fe6697fa700 1 -- 172.21.15.42:0/3595328612 wait complete. 2023-12-12T09:56:09.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.452+0000 7fe6697fa700 1 Processor -- start 2023-12-12T09:56:09.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.452+0000 7fe6697fa700 1 -- start start 2023-12-12T09:56:09.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.452+0000 7fe6697fa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe664101f00 0x7fe6641973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:09.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.452+0000 7fe6697fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664104830 0x7fe6641978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.452+0000 7fe6697fa700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664107160 0x7fe66419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.452+0000 7fe6697fa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419eee0 con 0x7fe664104830 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.452+0000 7fe6697fa700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419f020 con 0x7fe664107160 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.452+0000 7fe6697fa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66419f1b0 con 0x7fe664101f00 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6627fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664104830 0x7fe6641978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6627fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664104830 0x7fe6641978f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33120/0 (socket says 172.21.15.42:33120) 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6627fc700 1 -- 172.21.15.42:0/2625167838 learned_addr learned my addr 172.21.15.42:0/2625167838 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe662ffd700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe664101f00 0x7fe6641973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6637fe700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664107160 0x7fe66419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6627fc700 1 -- 172.21.15.42:0/2625167838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe664101f00 msgr2=0x7fe6641973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6627fc700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe664101f00 0x7fe6641973b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6627fc700 1 -- 172.21.15.42:0/2625167838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664107160 msgr2=0x7fe66419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6627fc700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664107160 0x7fe66419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6627fc700 1 -- 172.21.15.42:0/2625167838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540095f0 con 0x7fe664104830 2023-12-12T09:56:09.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6637fe700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664107160 0x7fe66419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:09.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6627fc700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664104830 0x7fe6641978f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe65800d980 tx=0x7fe65800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:09.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe64bfff700 1 -- 172.21.15.42:0/2625167838 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65800bb60 con 0x7fe664104830 2023-12-12T09:56:09.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe64bfff700 1 -- 172.21.15.42:0/2625167838 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe658007640 con 0x7fe664104830 2023-12-12T09:56:09.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66419f490 con 0x7fe664104830 2023-12-12T09:56:09.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe64bfff700 1 -- 172.21.15.42:0/2625167838 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe658004e90 con 0x7fe664104830 2023-12-12T09:56:09.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.453+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66419f9f0 con 0x7fe664104830 2023-12-12T09:56:09.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.454+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6640fd4b0 con 0x7fe664104830 2023-12-12T09:56:09.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.456+0000 7fe64bfff700 1 -- 172.21.15.42:0/2625167838 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe65800bcc0 con 0x7fe664104830 2023-12-12T09:56:09.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.456+0000 7fe64bfff700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe64c0a1500 0x7fe64c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:09.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.457+0000 7fe64bfff700 1 -- 172.21.15.42:0/2625167838 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe6580cca10 con 0x7fe664104830 2023-12-12T09:56:09.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.457+0000 7fe662ffd700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe64c0a1500 0x7fe64c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:09.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.457+0000 7fe662ffd700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe64c0a1500 0x7fe64c0a39c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe65400b5c0 tx=0x7fe654005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:09.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.460+0000 7fe64bfff700 1 -- 172.21.15.42:0/2625167838 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe658099f00 con 0x7fe664104830 2023-12-12T09:56:09.712 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.710+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe66419f620 con 0x7fe664104830 2023-12-12T09:56:09.712 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.710+0000 7fe64bfff700 1 -- 172.21.15.42:0/2625167838 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1328 (secure 0 0 0) 0x7fe658016540 con 0x7fe664104830 2023-12-12T09:56:09.712 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:09.713 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.712+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe64c0a1500 msgr2=0x7fe64c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.712+0000 7fe6697fa700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe64c0a1500 0x7fe64c0a39c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe65400b5c0 tx=0x7fe654005fb0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664104830 msgr2=0x7fe6641978f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664104830 0x7fe6641978f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe65800d980 tx=0x7fe65800dd40 comp rx=0 tx=0).stop 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 shutdown_connections 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe64c0a1500 0x7fe64c0a39c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe664107160 0x7fe66419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe664101f00 0x7fe6641973b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 --2- 172.21.15.42:0/2625167838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe664104830 0x7fe6641978f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 >> 172.21.15.42:0/2625167838 conn(0x7fe6640fb930 msgr2=0x7fe6641029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 shutdown_connections 2023-12-12T09:56:09.715 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:09.713+0000 7fe6697fa700 1 -- 172.21.15.42:0/2625167838 wait complete. 2023-12-12T09:56:09.716 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2023-12-12T09:56:09.803 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:09 smithi111 ceph-mon[102565]: Deploying daemon mds.k on smithi111 2023-12-12T09:56:09.803 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:09 smithi111 ceph-mon[102565]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:09.804 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 09:56:09 smithi111 podman[138610]: 2023-12-12T09:56:09.804 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 09:56:09 smithi111 ceph-mds[138650]: -- 172.21.15.111:0/720642802 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5621b95e9520 con 0x5621b95be800 2023-12-12T09:56:09.804 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 09:56:09 smithi111 bash[138610]: 6622f742790bec459d2e11da56733dc00a257dd817f06929feda0bfa0abac0a3 2023-12-12T09:56:09.804 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 09:56:09 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-k[138626]: starting mds.k at 2023-12-12T09:56:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:09 smithi042 ceph-mon[98894]: Deploying daemon mds.k on smithi111 2023-12-12T09:56:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:09 smithi042 ceph-mon[98894]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:10.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:09 smithi173 ceph-mon[103308]: Deploying daemon mds.k on smithi111 2023-12-12T09:56:10.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:09 smithi173 ceph-mon[103308]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:10.247 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 09:56:09 smithi111 systemd[1]: Started Ceph mds.k for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:10.862 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:10 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2625167838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:10.862 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:10.862 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:10.862 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:10.862 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:10.863 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:10.863 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:10 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:10 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2625167838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:10.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:10 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:11.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:10 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2625167838' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:10 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:11.403 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 09:56:11 smithi042 systemd[1]: Starting Ceph mds.j for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:11.728 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.868+0000 7f349f11a700 1 -- 172.21.15.42:0/2570432386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498104890 msgr2=0x7f3498106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.868+0000 7f349f11a700 1 --2- 172.21.15.42:0/2570432386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498104890 0x7f3498106c80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3484009990 tx=0x7f3484009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.868+0000 7f349f11a700 1 -- 172.21.15.42:0/2570432386 shutdown_connections 2023-12-12T09:56:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.868+0000 7f349f11a700 1 --2- 172.21.15.42:0/2570432386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3498101f60 0x7f3498104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.868+0000 7f349f11a700 1 --2- 172.21.15.42:0/2570432386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34981071c0 0x7f34981095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:11.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.868+0000 7f349f11a700 1 --2- 172.21.15.42:0/2570432386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498104890 0x7f3498106c80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.868+0000 7f349f11a700 1 -- 172.21.15.42:0/2570432386 >> 172.21.15.42:0/2570432386 conn(0x7f34980fb8f0 msgr2=0x7f34980fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.868+0000 7f349f11a700 1 -- 172.21.15.42:0/2570432386 shutdown_connections 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.868+0000 7f349f11a700 1 -- 172.21.15.42:0/2570432386 wait complete. 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349f11a700 1 Processor -- start 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349f11a700 1 -- start start 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349f11a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498101f60 0x7f34981973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349f11a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3498104890 0x7f3498197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349f11a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34981071c0 0x7f349819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349f11a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349819ef00 con 0x7f3498101f60 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349f11a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349819f040 con 0x7f3498104890 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349f11a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349819f1d0 con 0x7f34981071c0 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349ceb6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498101f60 0x7f34981973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:11.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349ceb6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498101f60 0x7f34981973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33130/0 (socket says 172.21.15.42:33130) 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349ceb6700 1 -- 172.21.15.42:0/4063939879 learned_addr learned my addr 172.21.15.42:0/4063939879 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f3497fff700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3498104890 0x7f3498197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349d6b7700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34981071c0 0x7f349819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349ceb6700 1 -- 172.21.15.42:0/4063939879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34981071c0 msgr2=0x7f349819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349ceb6700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34981071c0 0x7f349819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349ceb6700 1 -- 172.21.15.42:0/4063939879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3498104890 msgr2=0x7f3498197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349ceb6700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3498104890 0x7f3498197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349ceb6700 1 -- 172.21.15.42:0/4063939879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840095f0 con 0x7f3498101f60 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f3497fff700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3498104890 0x7f3498197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.869+0000 7f349ceb6700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498101f60 0x7f34981973d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c920 tx=0x7f348c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.870+0000 7f3495ffb700 1 -- 172.21.15.42:0/4063939879 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c007b00 con 0x7f3498101f60 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.870+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349819f4b0 con 0x7f3498101f60 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.870+0000 7f3495ffb700 1 -- 172.21.15.42:0/4063939879 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f348c004cf0 con 0x7f3498101f60 2023-12-12T09:56:11.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.870+0000 7f3495ffb700 1 -- 172.21.15.42:0/4063939879 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c005610 con 0x7f3498101f60 2023-12-12T09:56:11.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.870+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349819f9c0 con 0x7f3498101f60 2023-12-12T09:56:11.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.871+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34980fd470 con 0x7f3498101f60 2023-12-12T09:56:11.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.872+0000 7f3495ffb700 1 -- 172.21.15.42:0/4063939879 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f348c004750 con 0x7f3498101f60 2023-12-12T09:56:11.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.872+0000 7f3495ffb700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f34880a1480 0x7f34880a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:11.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.872+0000 7f3495ffb700 1 -- 172.21.15.42:0/4063939879 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f348c0cc810 con 0x7f3498101f60 2023-12-12T09:56:11.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.872+0000 7f3497fff700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f34880a1480 0x7f34880a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:11.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.873+0000 7f3497fff700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f34880a1480 0x7f34880a3940 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3484006010 tx=0x7f34840037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:11.879 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:11.877+0000 7f3495ffb700 1 -- 172.21.15.42:0/4063939879 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f348c096310 con 0x7f3498101f60 2023-12-12T09:56:11.960 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 09:56:11 smithi042 podman[150115]: 2023-12-12T09:56:11.960 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:11 smithi042 ceph-mon[98894]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:11.960 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:11 smithi042 ceph-mon[98894]: Deploying daemon mds.j on smithi042 2023-12-12T09:56:11.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:11 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6832/1539682179,v1:172.21.15.111:6833/1539682179] up:boot 2023-12-12T09:56:11.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:11 smithi042 ceph-mon[98894]: fsmap 2 up:standby 2023-12-12T09:56:11.961 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:11 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-12T09:56:11.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:11 smithi111 ceph-mon[102565]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:11.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:11 smithi111 ceph-mon[102565]: Deploying daemon mds.j on smithi042 2023-12-12T09:56:11.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:11 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6832/1539682179,v1:172.21.15.111:6833/1539682179] up:boot 2023-12-12T09:56:11.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:11 smithi111 ceph-mon[102565]: fsmap 2 up:standby 2023-12-12T09:56:11.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:11 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-12T09:56:12.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:11 smithi173 ceph-mon[103308]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:12.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:11 smithi173 ceph-mon[103308]: Deploying daemon mds.j on smithi042 2023-12-12T09:56:12.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:11 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6832/1539682179,v1:172.21.15.111:6833/1539682179] up:boot 2023-12-12T09:56:12.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:11 smithi173 ceph-mon[103308]: fsmap 2 up:standby 2023-12-12T09:56:12.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:11 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-12T09:56:12.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.148+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3498068230 con 0x7f3498101f60 2023-12-12T09:56:12.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.148+0000 7f3495ffb700 1 -- 172.21.15.42:0/4063939879 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2122 (secure 0 0 0) 0x7f348c01d860 con 0x7f3498101f60 2023-12-12T09:56:12.150 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:12.150 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-12T09:56:12.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f34880a1480 msgr2=0x7f34880a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:12.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f34880a1480 0x7f34880a3940 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3484006010 tx=0x7f34840037b0 comp rx=0 tx=0).stop 2023-12-12T09:56:12.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498101f60 msgr2=0x7f34981973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:12.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498101f60 0x7f34981973d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c920 tx=0x7f348c00cce0 comp rx=0 tx=0).stop 2023-12-12T09:56:12.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 shutdown_connections 2023-12-12T09:56:12.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f34880a1480 0x7f34880a3940 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:12.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34981071c0 0x7f349819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:12.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3498101f60 0x7f34981973d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:12.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 --2- 172.21.15.42:0/4063939879 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3498104890 0x7f3498197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:12.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 >> 172.21.15.42:0/4063939879 conn(0x7f34980fb8f0 msgr2=0x7f34981029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:12.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 shutdown_connections 2023-12-12T09:56:12.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:12.150+0000 7f349f11a700 1 -- 172.21.15.42:0/4063939879 wait complete. 2023-12-12T09:56:12.153 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 3 2023-12-12T09:56:12.403 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 09:56:11 smithi042 bash[150115]: 2ecb72fc0275ced50e1b017bbc7a908986dd06ca7b3b336d897880606e849484 2023-12-12T09:56:12.403 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 09:56:11 smithi042 ceph-mds[150197]: -- 172.21.15.42:0/2972775624 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ce29d69520 con 0x55ce29d3f000 2023-12-12T09:56:12.403 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 09:56:11 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-j[150151]: starting mds.j at 2023-12-12T09:56:12.403 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 09:56:11 smithi042 systemd[1]: Started Ceph mds.j for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: Deploying daemon mds.d on smithi042 2023-12-12T09:56:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/4063939879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:13.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.42:6834/4073397880,v1:172.21.15.42:6835/4073397880] up:boot 2023-12-12T09:56:13.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: fsmap 3 up:standby 2023-12-12T09:56:13.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:13 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-12T09:56:13.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:13.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: Deploying daemon mds.d on smithi042 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/4063939879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.42:6834/4073397880,v1:172.21.15.42:6835/4073397880] up:boot 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: fsmap 3 up:standby 2023-12-12T09:56:13.337 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:13 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-12T09:56:13.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:13.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:13.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:13.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:13.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:13.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:13.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:13.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: Deploying daemon mds.d on smithi042 2023-12-12T09:56:13.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/4063939879' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:13.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.42:6834/4073397880,v1:172.21.15.42:6835/4073397880] up:boot 2023-12-12T09:56:13.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: fsmap 3 up:standby 2023-12-12T09:56:13.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:13 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-12T09:56:13.903 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 09:56:13 smithi042 systemd[1]: Starting Ceph mds.d for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:14.166 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:14.193 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 09:56:13 smithi042 podman[150497]: 2023-12-12T09:56:14.324 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.322+0000 7f67049c0700 1 -- 172.21.15.42:0/1230876750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6700103030 msgr2=0x7f67001098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:14.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.322+0000 7f67049c0700 1 --2- 172.21.15.42:0/1230876750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6700103030 0x7f67001098e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f66f800b0d0 tx=0x7f66f800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.322+0000 7f67049c0700 1 -- 172.21.15.42:0/1230876750 shutdown_connections 2023-12-12T09:56:14.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.322+0000 7f67049c0700 1 --2- 172.21.15.42:0/1230876750 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f67001014a0 0x7f67001018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.322+0000 7f67049c0700 1 --2- 172.21.15.42:0/1230876750 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6700103030 0x7f67001098e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.322+0000 7f67049c0700 1 --2- 172.21.15.42:0/1230876750 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6700102690 0x7f6700102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.322+0000 7f67049c0700 1 -- 172.21.15.42:0/1230876750 >> 172.21.15.42:0/1230876750 conn(0x7f67000fca10 msgr2=0x7f67000fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:14.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.322+0000 7f67049c0700 1 -- 172.21.15.42:0/1230876750 shutdown_connections 2023-12-12T09:56:14.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.322+0000 7f67049c0700 1 -- 172.21.15.42:0/1230876750 wait complete. 2023-12-12T09:56:14.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.323+0000 7f67049c0700 1 Processor -- start 2023-12-12T09:56:14.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.323+0000 7f67049c0700 1 -- start start 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.323+0000 7f67049c0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67001014a0 0x7f6700075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.323+0000 7f67049c0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6700102690 0x7f67000739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.323+0000 7f67049c0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6700103030 0x7f6700073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.323+0000 7f67049c0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000746a0 con 0x7f67001014a0 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.323+0000 7f67049c0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6700074810 con 0x7f6700103030 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.323+0000 7f67049c0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67001a1550 con 0x7f6700102690 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fe59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67001014a0 0x7f6700075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fe59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67001014a0 0x7f6700075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33172/0 (socket says 172.21.15.42:33172) 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fe59c700 1 -- 172.21.15.42:0/2097851254 learned_addr learned my addr 172.21.15.42:0/2097851254 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fed9d700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6700103030 0x7f6700073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fdd9b700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6700102690 0x7f67000739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fe59c700 1 -- 172.21.15.42:0/2097851254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6700102690 msgr2=0x7f67000739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fe59c700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6700102690 0x7f67000739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fe59c700 1 -- 172.21.15.42:0/2097851254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6700103030 msgr2=0x7f6700073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:14.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fe59c700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6700103030 0x7f6700073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fe59c700 1 -- 172.21.15.42:0/2097851254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f8009d00 con 0x7f67001014a0 2023-12-12T09:56:14.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66fe59c700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67001014a0 0x7f6700075330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f66f000e970 tx=0x7f66f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:14.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66ef7fe700 1 -- 172.21.15.42:0/2097851254 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0009790 con 0x7f67001014a0 2023-12-12T09:56:14.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66ef7fe700 1 -- 172.21.15.42:0/2097851254 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66f000c780 con 0x7f67001014a0 2023-12-12T09:56:14.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67001a17e0 con 0x7f67001014a0 2023-12-12T09:56:14.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f66ef7fe700 1 -- 172.21.15.42:0/2097851254 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0003be0 con 0x7f67001014a0 2023-12-12T09:56:14.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.324+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67001a1d40 con 0x7f67001014a0 2023-12-12T09:56:14.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.325+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67001088a0 con 0x7f67001014a0 2023-12-12T09:56:14.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.327+0000 7f66ef7fe700 1 -- 172.21.15.42:0/2097851254 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f66f0003d60 con 0x7f67001014a0 2023-12-12T09:56:14.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.327+0000 7f66ef7fe700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f66e80a1500 0x7f66e80a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:14.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.327+0000 7f66fdd9b700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f66e80a1500 0x7f66e80a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:14.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.327+0000 7f66ef7fe700 1 -- 172.21.15.42:0/2097851254 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f66f0014070 con 0x7f67001014a0 2023-12-12T09:56:14.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.328+0000 7f66fdd9b700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f66e80a1500 0x7f66e80a39c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f66f4007be0 tx=0x7f66f40073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:14.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.331+0000 7f66ef7fe700 1 -- 172.21.15.42:0/2097851254 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f66f0096590 con 0x7f67001014a0 2023-12-12T09:56:14.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.581+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6700068230 con 0x7f67001014a0 2023-12-12T09:56:14.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.582+0000 7f66ef7fe700 1 -- 172.21.15.42:0/2097851254 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f66f0094e60 con 0x7f67001014a0 2023-12-12T09:56:14.584 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:14.584 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-12T09:56:14.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f66e80a1500 msgr2=0x7f66e80a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:14.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f66e80a1500 0x7f66e80a39c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f66f4007be0 tx=0x7f66f40073d0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67001014a0 msgr2=0x7f6700075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:14.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67001014a0 0x7f6700075330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f66f000e970 tx=0x7f66f000ec80 comp rx=0 tx=0).stop 2023-12-12T09:56:14.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 shutdown_connections 2023-12-12T09:56:14.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f66e80a1500 0x7f66e80a39c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6700103030 0x7f6700073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f67001014a0 0x7f6700075330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 --2- 172.21.15.42:0/2097851254 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6700102690 0x7f67000739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:14.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 >> 172.21.15.42:0/2097851254 conn(0x7f67000fca10 msgr2=0x7f67001059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:14.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 shutdown_connections 2023-12-12T09:56:14.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:14.584+0000 7f67049c0700 1 -- 172.21.15.42:0/2097851254 wait complete. 2023-12-12T09:56:14.587 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 5 2023-12-12T09:56:14.597 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 09:56:14 smithi042 bash[150497]: ca9a3af31db239c7948805f85f61c303c8f251c0a171c980a495b7bbc0f13566 2023-12-12T09:56:14.597 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 09:56:14 smithi042 ceph-mds[150540]: -- 172.21.15.42:0/2959908491 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5600e1ee3520 con 0x5600e1eb9000 2023-12-12T09:56:14.597 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 09:56:14 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-d[150514]: starting mds.d at 2023-12-12T09:56:14.597 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 09:56:14 smithi042 systemd[1]: Started Ceph mds.d for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:15.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:15.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.42:6836/3743179195,v1:172.21.15.42:6837/3743179195] up:boot 2023-12-12T09:56:15.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: fsmap 4 up:standby 2023-12-12T09:56:15.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-12T09:56:15.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:15.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:15.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:15.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:15.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:15.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:15.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: Deploying daemon mds.e on smithi111 2023-12-12T09:56:15.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:15 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2097851254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.42:6836/3743179195,v1:172.21.15.42:6837/3743179195] up:boot 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: fsmap 4 up:standby 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:15.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: Deploying daemon mds.e on smithi111 2023-12-12T09:56:15.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:15 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2097851254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:15.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:15.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.42:6836/3743179195,v1:172.21.15.42:6837/3743179195] up:boot 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: fsmap 4 up:standby 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: Deploying daemon mds.e on smithi111 2023-12-12T09:56:15.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:15 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2097851254' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:16.045 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 09:56:15 smithi111 systemd[1]: Starting Ceph mds.e for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:16.345 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 09:56:16 smithi111 podman[138949]: 2023-12-12T09:56:16.345 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 09:56:16 smithi111 ceph-mds[138989]: -- 172.21.15.111:0/1884212029 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cc6ae61520 con 0x55cc6ae37000 2023-12-12T09:56:16.345 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 09:56:16 smithi111 bash[138949]: 98f0622a59961c4260834846e44dbcfb9297cac7b6e18a3849e7350faedb234e 2023-12-12T09:56:16.345 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 09:56:16 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-e[138965]: starting mds.e at 2023-12-12T09:56:16.600 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:16.740 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.738+0000 7f3953102700 1 -- 172.21.15.42:0/2060111427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c100380 msgr2=0x7f394c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.738+0000 7f3953102700 1 --2- 172.21.15.42:0/2060111427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c100380 0x7f394c1007a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f39400099c0 tx=0x7f3940009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.738+0000 7f3953102700 1 -- 172.21.15.42:0/2060111427 shutdown_connections 2023-12-12T09:56:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.738+0000 7f3953102700 1 --2- 172.21.15.42:0/2060111427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f394c101f10 0x7f394c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.738+0000 7f3953102700 1 --2- 172.21.15.42:0/2060111427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c100380 0x7f394c1007a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.738+0000 7f3953102700 1 --2- 172.21.15.42:0/2060111427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394c101570 0x7f394c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.738+0000 7f3953102700 1 -- 172.21.15.42:0/2060111427 >> 172.21.15.42:0/2060111427 conn(0x7f394c0fb8f0 msgr2=0x7f394c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.738+0000 7f3953102700 1 -- 172.21.15.42:0/2060111427 shutdown_connections 2023-12-12T09:56:16.741 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.738+0000 7f3953102700 1 -- 172.21.15.42:0/2060111427 wait complete. 2023-12-12T09:56:16.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f3953102700 1 Processor -- start 2023-12-12T09:56:16.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f3953102700 1 -- start start 2023-12-12T09:56:16.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f3953102700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f394c100380 0x7f394c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f3953102700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394c101570 0x7f394c196810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f3953102700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c101f10 0x7f394c19d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f3953102700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c19de00 con 0x7f394c101f10 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f3953102700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c19df40 con 0x7f394c101570 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f3953102700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c19e0d0 con 0x7f394c100380 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f395169f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c101f10 0x7f394c19d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f395169f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c101f10 0x7f394c19d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33190/0 (socket says 172.21.15.42:33190) 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f3950e9e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f394c100380 0x7f394c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f395169f700 1 -- 172.21.15.42:0/2147379393 learned_addr learned my addr 172.21.15.42:0/2147379393 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f394bfff700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394c101570 0x7f394c196810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f395169f700 1 -- 172.21.15.42:0/2147379393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f394c100380 msgr2=0x7f394c1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f395169f700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f394c100380 0x7f394c1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f395169f700 1 -- 172.21.15.42:0/2147379393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394c101570 msgr2=0x7f394c196810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f395169f700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394c101570 0x7f394c196810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:16.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.739+0000 7f395169f700 1 -- 172.21.15.42:0/2147379393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39400095f0 con 0x7f394c101f10 2023-12-12T09:56:16.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.740+0000 7f395169f700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c101f10 0x7f394c19d830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3944007ee0 tx=0x7f394400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:16.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.740+0000 7f3949ffb700 1 -- 172.21.15.42:0/2147379393 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944017070 con 0x7f394c101f10 2023-12-12T09:56:16.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.740+0000 7f3949ffb700 1 -- 172.21.15.42:0/2147379393 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f394400f040 con 0x7f394c101f10 2023-12-12T09:56:16.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.740+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c19e3b0 con 0x7f394c101f10 2023-12-12T09:56:16.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.740+0000 7f3949ffb700 1 -- 172.21.15.42:0/2147379393 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944005580 con 0x7f394c101f10 2023-12-12T09:56:16.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.740+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c19e910 con 0x7f394c101f10 2023-12-12T09:56:16.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.741+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c1077a0 con 0x7f394c101f10 2023-12-12T09:56:16.746 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.742+0000 7f3949ffb700 1 -- 172.21.15.42:0/2147379393 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f394400db20 con 0x7f394c101f10 2023-12-12T09:56:16.746 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.743+0000 7f3949ffb700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1500 0x7f393c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:16.746 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.743+0000 7f3949ffb700 1 -- 172.21.15.42:0/2147379393 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f39440cbef0 con 0x7f394c101f10 2023-12-12T09:56:16.746 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.743+0000 7f3950e9e700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1500 0x7f393c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:16.746 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.743+0000 7f3950e9e700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1500 0x7f393c0a39c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3940003820 tx=0x7f3940005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:16.747 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 09:56:16 smithi111 systemd[1]: Started Ceph mds.e for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:16.749 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.747+0000 7f3949ffb700 1 -- 172.21.15.42:0/2147379393 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f39440959f0 con 0x7f394c101f10 2023-12-12T09:56:16.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.998+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f394c068230 con 0x7f394c101f10 2023-12-12T09:56:17.000 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:16.998+0000 7f3949ffb700 1 -- 172.21.15.42:0/2147379393 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3704 (secure 0 0 0) 0x7f3944099010 con 0x7f394c101f10 2023-12-12T09:56:17.000 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:17.000 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-12T09:56:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1500 msgr2=0x7f393c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1500 0x7f393c0a39c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3940003820 tx=0x7f3940005920 comp rx=0 tx=0).stop 2023-12-12T09:56:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c101f10 msgr2=0x7f394c19d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:17.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c101f10 0x7f394c19d830 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3944007ee0 tx=0x7f394400d3b0 comp rx=0 tx=0).stop 2023-12-12T09:56:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 shutdown_connections 2023-12-12T09:56:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1500 0x7f393c0a39c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f394c101f10 0x7f394c19d830 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f394c100380 0x7f394c1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 --2- 172.21.15.42:0/2147379393 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f394c101570 0x7f394c196810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.000+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 >> 172.21.15.42:0/2147379393 conn(0x7f394c0fb8f0 msgr2=0x7f394c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:17.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.001+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 shutdown_connections 2023-12-12T09:56:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:17.001+0000 7f3953102700 1 -- 172.21.15.42:0/2147379393 wait complete. 2023-12-12T09:56:17.004 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 5 2023-12-12T09:56:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:17.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:17.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:17.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:17.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:17.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:17.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: Deploying daemon mds.f on smithi173 2023-12-12T09:56:17.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2147379393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:17.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6834/4127941301,v1:172.21.15.111:6835/4127941301] up:boot 2023-12-12T09:56:17.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: fsmap 5 up:standby 2023-12-12T09:56:17.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:17 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-12T09:56:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:17.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: Deploying daemon mds.f on smithi173 2023-12-12T09:56:17.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2147379393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:17.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6834/4127941301,v1:172.21.15.111:6835/4127941301] up:boot 2023-12-12T09:56:17.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: fsmap 5 up:standby 2023-12-12T09:56:17.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:17 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-12T09:56:17.757 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:17.757 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:17.757 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:17.757 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:17.758 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:17.758 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:17.758 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:17.758 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: Deploying daemon mds.f on smithi173 2023-12-12T09:56:17.758 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2147379393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:17.758 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6834/4127941301,v1:172.21.15.111:6835/4127941301] up:boot 2023-12-12T09:56:17.758 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: fsmap 5 up:standby 2023-12-12T09:56:17.758 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:17 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-12T09:56:18.073 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 09:56:17 smithi173 systemd[1]: Starting Ceph mds.f for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:18.536 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 09:56:18 smithi173 podman[139805]: 2023-12-12T09:56:18.822 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 09:56:18 smithi173 ceph-mds[139845]: -- 172.21.15.173:0/3050370368 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d78c593520 con 0x55d78c568c00 2023-12-12T09:56:18.823 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 09:56:18 smithi173 bash[139805]: 1b95c1c40ddecdddf605483151d646003c50d413bfa1023b9514efdb5222e571 2023-12-12T09:56:18.823 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 09:56:18 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-f[139821]: starting mds.f at 2023-12-12T09:56:18.823 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 09:56:18 smithi173 systemd[1]: Started Ceph mds.f for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:19.015 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.154+0000 7f8a64d20700 1 -- 172.21.15.42:0/3478074261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60101fc0 msgr2=0x7f8a601043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.154+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3478074261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60101fc0 0x7f8a601043b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8a50009990 tx=0x7f8a50009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.154+0000 7f8a64d20700 1 -- 172.21.15.42:0/3478074261 shutdown_connections 2023-12-12T09:56:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.154+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3478074261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a60107220 0x7f8a60109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.154+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3478074261 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60101fc0 0x7f8a601043b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.154+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3478074261 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a601048f0 0x7f8a60106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.160 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.154+0000 7f8a64d20700 1 -- 172.21.15.42:0/3478074261 >> 172.21.15.42:0/3478074261 conn(0x7f8a600fb8f0 msgr2=0x7f8a600fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.154+0000 7f8a64d20700 1 -- 172.21.15.42:0/3478074261 shutdown_connections 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.154+0000 7f8a64d20700 1 -- 172.21.15.42:0/3478074261 wait complete. 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a64d20700 1 Processor -- start 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a64d20700 1 -- start start 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a64d20700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a60101fc0 0x7f8a601973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a64d20700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a601048f0 0x7f8a601978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a64d20700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60107220 0x7f8a6019e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a64d20700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6019eed0 con 0x7f8a60107220 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a64d20700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6019f010 con 0x7f8a60101fc0 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a64d20700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6019f1a0 con 0x7f8a601048f0 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a5ed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60107220 0x7f8a6019e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a5e59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a60101fc0 0x7f8a601973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a5dd9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a601048f0 0x7f8a601978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:19.161 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a5ed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60107220 0x7f8a6019e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33204/0 (socket says 172.21.15.42:33204) 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.155+0000 7f8a5ed9d700 1 -- 172.21.15.42:0/3294010624 learned_addr learned my addr 172.21.15.42:0/3294010624 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a5ed9d700 1 -- 172.21.15.42:0/3294010624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a601048f0 msgr2=0x7f8a601978e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a5ed9d700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a601048f0 0x7f8a601978e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a5ed9d700 1 -- 172.21.15.42:0/3294010624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a60101fc0 msgr2=0x7f8a601973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a5ed9d700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a60101fc0 0x7f8a601973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a5ed9d700 1 -- 172.21.15.42:0/3294010624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a500095f0 con 0x7f8a60107220 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a5e59c700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a60101fc0 0x7f8a601973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a5dd9b700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a601048f0 0x7f8a601978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a5ed9d700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60107220 0x7f8a6019e900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8a58007ee0 tx=0x7f8a5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a577fe700 1 -- 172.21.15.42:0/3294010624 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58017070 con 0x7f8a60107220 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a577fe700 1 -- 172.21.15.42:0/3294010624 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a5800f040 con 0x7f8a60107220 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a577fe700 1 -- 172.21.15.42:0/3294010624 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58005580 con 0x7f8a60107220 2023-12-12T09:56:19.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6019f480 con 0x7f8a60107220 2023-12-12T09:56:19.163 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.156+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6019f940 con 0x7f8a60107220 2023-12-12T09:56:19.163 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.157+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a600fd470 con 0x7f8a60107220 2023-12-12T09:56:19.163 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.158+0000 7f8a577fe700 1 -- 172.21.15.42:0/3294010624 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8a5800db20 con 0x7f8a60107220 2023-12-12T09:56:19.163 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.159+0000 7f8a577fe700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a4c0a1480 0x7f8a4c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:19.163 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.159+0000 7f8a577fe700 1 -- 172.21.15.42:0/3294010624 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8a580cc630 con 0x7f8a60107220 2023-12-12T09:56:19.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.159+0000 7f8a5e59c700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a4c0a1480 0x7f8a4c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:19.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.159+0000 7f8a5e59c700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a4c0a1480 0x7f8a4c0a3940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000bc60 tx=0x7f8a50003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:19.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.162+0000 7f8a577fe700 1 -- 172.21.15.42:0/3294010624 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8a58096130 con 0x7f8a60107220 2023-12-12T09:56:19.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.412+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a60068230 con 0x7f8a60107220 2023-12-12T09:56:19.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.413+0000 7f8a577fe700 1 -- 172.21.15.42:0/3294010624 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5292 (secure 0 0 0) 0x7f8a58099750 con 0x7f8a60107220 2023-12-12T09:56:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:19.415 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a4c0a1480 msgr2=0x7f8a4c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a4c0a1480 0x7f8a4c0a3940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000bc60 tx=0x7f8a50003790 comp rx=0 tx=0).stop 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60107220 msgr2=0x7f8a6019e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60107220 0x7f8a6019e900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8a58007ee0 tx=0x7f8a5800d3b0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 shutdown_connections 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8a4c0a1480 0x7f8a4c0a3940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8a60101fc0 0x7f8a601973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a60107220 0x7f8a6019e900 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 --2- 172.21.15.42:0/3294010624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a601048f0 0x7f8a601978e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 >> 172.21.15.42:0/3294010624 conn(0x7f8a600fb8f0 msgr2=0x7f8a60102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 shutdown_connections 2023-12-12T09:56:19.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:19.415+0000 7f8a64d20700 1 -- 172.21.15.42:0/3294010624 wait complete. 2023-12-12T09:56:19.418 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 7 2023-12-12T09:56:19.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:19.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:19.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:19.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:19.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:19.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:19.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:19.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] up:boot 2023-12-12T09:56:19.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: fsmap 6 up:standby 2023-12-12T09:56:19.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-12T09:56:19.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:19 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3294010624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:19.979 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:19.979 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:19.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:19.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:19.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:19.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:19.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:19.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] up:boot 2023-12-12T09:56:19.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: fsmap 6 up:standby 2023-12-12T09:56:19.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-12T09:56:19.980 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:19 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3294010624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:19.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:19.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] up:boot 2023-12-12T09:56:19.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: fsmap 6 up:standby 2023-12-12T09:56:19.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-12T09:56:19.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:19 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3294010624' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:20.323 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 09:56:20 smithi173 systemd[1]: Starting Ceph mds.c for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:20.779 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 09:56:20 smithi173 podman[140143]: 2023-12-12T09:56:20.779 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:20 smithi173 ceph-mon[103308]: Deploying daemon mds.c on smithi173 2023-12-12T09:56:20.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:20 smithi042 ceph-mon[98894]: Deploying daemon mds.c on smithi173 2023-12-12T09:56:20.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:20 smithi111 ceph-mon[102565]: Deploying daemon mds.c on smithi173 2023-12-12T09:56:21.073 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 09:56:20 smithi173 ceph-mds[140183]: -- 172.21.15.173:0/493046921 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558665a83520 con 0x558665a58c00 2023-12-12T09:56:21.073 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 09:56:20 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-c[140159]: starting mds.c at 2023-12-12T09:56:21.073 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 09:56:20 smithi173 bash[140143]: 0b85d811b68962476f208501419ba9135b801edaeb5faabd93f0a262fd1a225b 2023-12-12T09:56:21.073 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 09:56:20 smithi173 systemd[1]: Started Ceph mds.c for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:21.430 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:21.571 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.569+0000 7f8231e2f700 1 -- 172.21.15.42:0/2926348273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1026a0 msgr2=0x7f822c102b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.569+0000 7f8231e2f700 1 --2- 172.21.15.42:0/2926348273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1026a0 0x7f822c102b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f821c0099c0 tx=0x7f821c009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.569+0000 7f8231e2f700 1 -- 172.21.15.42:0/2926348273 shutdown_connections 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.569+0000 7f8231e2f700 1 --2- 172.21.15.42:0/2926348273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c103040 0x7f822c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.569+0000 7f8231e2f700 1 --2- 172.21.15.42:0/2926348273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f822c1014b0 0x7f822c1018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.569+0000 7f8231e2f700 1 --2- 172.21.15.42:0/2926348273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1026a0 0x7f822c102b00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.569+0000 7f8231e2f700 1 -- 172.21.15.42:0/2926348273 >> 172.21.15.42:0/2926348273 conn(0x7f822c0fca20 msgr2=0x7f822c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.569+0000 7f8231e2f700 1 -- 172.21.15.42:0/2926348273 shutdown_connections 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.569+0000 7f8231e2f700 1 -- 172.21.15.42:0/2926348273 wait complete. 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f8231e2f700 1 Processor -- start 2023-12-12T09:56:21.572 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f8231e2f700 1 -- start start 2023-12-12T09:56:21.574 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f8231e2f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1014b0 0x7f822c075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:21.574 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f8231e2f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f822c1026a0 0x7f822c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:21.574 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f8231e2f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c103040 0x7f822c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:21.574 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f8231e2f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0746a0 con 0x7f822c1014b0 2023-12-12T09:56:21.574 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f8231e2f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c074810 con 0x7f822c103040 2023-12-12T09:56:21.574 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f8231e2f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c1a1460 con 0x7f822c1026a0 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1014b0 0x7f822c075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1014b0 0x7f822c075330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33214/0 (socket says 172.21.15.42:33214) 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822b7fe700 1 -- 172.21.15.42:0/351829577 learned_addr learned my addr 172.21.15.42:0/351829577 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822bfff700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c103040 0x7f822c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822affd700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f822c1026a0 0x7f822c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822b7fe700 1 -- 172.21.15.42:0/351829577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f822c1026a0 msgr2=0x7f822c0739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822b7fe700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f822c1026a0 0x7f822c0739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822b7fe700 1 -- 172.21.15.42:0/351829577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c103040 msgr2=0x7f822c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822b7fe700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c103040 0x7f822c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.570+0000 7f822b7fe700 1 -- 172.21.15.42:0/351829577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0095f0 con 0x7f822c1014b0 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.571+0000 7f822b7fe700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1014b0 0x7f822c075330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f822000d980 tx=0x7f822000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.571+0000 7f8228ff9700 1 -- 172.21.15.42:0/351829577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220004020 con 0x7f822c1014b0 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.571+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c1a16f0 con 0x7f822c1014b0 2023-12-12T09:56:21.575 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.571+0000 7f8228ff9700 1 -- 172.21.15.42:0/351829577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f822000f690 con 0x7f822c1014b0 2023-12-12T09:56:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.571+0000 7f8228ff9700 1 -- 172.21.15.42:0/351829577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220003a60 con 0x7f822c1014b0 2023-12-12T09:56:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.571+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c1a1c50 con 0x7f822c1014b0 2023-12-12T09:56:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.572+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c108860 con 0x7f822c1014b0 2023-12-12T09:56:21.576 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.573+0000 7f8228ff9700 1 -- 172.21.15.42:0/351829577 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f822000fb00 con 0x7f822c1014b0 2023-12-12T09:56:21.577 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.574+0000 7f8228ff9700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f82140a1480 0x7f82140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:21.577 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.574+0000 7f822affd700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f82140a1480 0x7f82140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:21.577 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.574+0000 7f8228ff9700 1 -- 172.21.15.42:0/351829577 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f82200cc3f0 con 0x7f822c1014b0 2023-12-12T09:56:21.577 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.574+0000 7f822affd700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f82140a1480 0x7f82140a3940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f821c003730 tx=0x7f821c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:21.580 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.578+0000 7f8228ff9700 1 -- 172.21.15.42:0/351829577 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8220095ef0 con 0x7f822c1014b0 2023-12-12T09:56:21.835 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.833+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f822c10cfd0 con 0x7f822c1014b0 2023-12-12T09:56:21.836 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.834+0000 7f8228ff9700 1 -- 172.21.15.42:0/351829577 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6086 (secure 0 0 0) 0x7f8220099510 con 0x7f822c1014b0 2023-12-12T09:56:21.836 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:21.836 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-12-12T09:56:21.838 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f82140a1480 msgr2=0x7f82140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:21.838 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f82140a1480 0x7f82140a3940 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f821c003730 tx=0x7f821c00b540 comp rx=0 tx=0).stop 2023-12-12T09:56:21.838 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1014b0 msgr2=0x7f822c075330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:21.838 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1014b0 0x7f822c075330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f822000d980 tx=0x7f822000dc90 comp rx=0 tx=0).stop 2023-12-12T09:56:21.838 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 shutdown_connections 2023-12-12T09:56:21.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f82140a1480 0x7f82140a3940 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f822c103040 0x7f822c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f822c1014b0 0x7f822c075330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 --2- 172.21.15.42:0/351829577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f822c1026a0 0x7f822c0739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:21.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 >> 172.21.15.42:0/351829577 conn(0x7f822c0fca20 msgr2=0x7f822c1059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:21.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 shutdown_connections 2023-12-12T09:56:21.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:21.836+0000 7f8231e2f700 1 -- 172.21.15.42:0/351829577 wait complete. 2023-12-12T09:56:21.839 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2023-12-12T09:56:21.849 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:21 smithi042 ceph-mon[98894]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:21.849 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:21.850 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:21.850 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:21.850 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:21.850 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:21.850 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:21 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:21.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:21 smithi111 ceph-mon[102565]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:21.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:21.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:21.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:21.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:21.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:21.959 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:21 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:22.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:21 smithi173 ceph-mon[103308]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:22.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:22.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:22.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:22.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:22.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:22.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:21 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:22.645 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 09:56:22 smithi111 systemd[1]: Starting Ceph mds.h for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:22.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:22 smithi042 ceph-mon[98894]: Deploying daemon mds.h on smithi111 2023-12-12T09:56:22.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:22 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] up:boot 2023-12-12T09:56:22.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:22 smithi042 ceph-mon[98894]: fsmap 7 up:standby 2023-12-12T09:56:22.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:22 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-12T09:56:22.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:22 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/351829577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:22.932 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 09:56:22 smithi111 podman[139292]: 2023-12-12T09:56:22.933 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:22 smithi111 ceph-mon[102565]: Deploying daemon mds.h on smithi111 2023-12-12T09:56:22.933 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:22 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] up:boot 2023-12-12T09:56:22.933 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:22 smithi111 ceph-mon[102565]: fsmap 7 up:standby 2023-12-12T09:56:22.933 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:22 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-12T09:56:22.933 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:22 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/351829577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:22 smithi173 ceph-mon[103308]: Deploying daemon mds.h on smithi111 2023-12-12T09:56:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:22 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] up:boot 2023-12-12T09:56:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:22 smithi173 ceph-mon[103308]: fsmap 7 up:standby 2023-12-12T09:56:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:22 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-12T09:56:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:22 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/351829577' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:23.247 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 09:56:22 smithi111 bash[139292]: 9484396f74d30e75962d16bc2bdfc6da24e3c821e59f67c96c47556c7c711bfe 2023-12-12T09:56:23.247 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 09:56:22 smithi111 ceph-mds[139333]: -- 172.21.15.111:0/3676790449 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a74c52d520 con 0x55a74c502800 2023-12-12T09:56:23.247 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 09:56:22 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-h[139308]: starting mds.h at 2023-12-12T09:56:23.247 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 09:56:22 smithi111 systemd[1]: Started Ceph mds.h for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:23.851 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:23.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:23 smithi042 ceph-mon[98894]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:23.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:23 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:23.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:23 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:23.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:23 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:23.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:23 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:23.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:23 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:23.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:23 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:23.961 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:23 smithi173 ceph-mon[103308]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:23.961 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:23 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:23.961 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:23 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:23.961 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:23 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:23.961 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:23 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:23.961 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:23 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:23.961 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:23 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.990+0000 7fde20dc3700 1 -- 172.21.15.42:0/1600091514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 msgr2=0x7fde1c1042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:23.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.990+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1600091514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 0x7fde1c1042f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fde0c009990 tx=0x7fde0c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:23.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.990+0000 7fde20dc3700 1 -- 172.21.15.42:0/1600091514 shutdown_connections 2023-12-12T09:56:23.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.990+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1600091514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c107160 0x7fde1c1095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:23.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.990+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1600091514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 0x7fde1c1042f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:23.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.990+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1600091514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde1c104830 0x7fde1c106c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:23.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.990+0000 7fde20dc3700 1 -- 172.21.15.42:0/1600091514 >> 172.21.15.42:0/1600091514 conn(0x7fde1c0fb8f0 msgr2=0x7fde1c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:23.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.990+0000 7fde20dc3700 1 -- 172.21.15.42:0/1600091514 shutdown_connections 2023-12-12T09:56:23.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.990+0000 7fde20dc3700 1 -- 172.21.15.42:0/1600091514 wait complete. 2023-12-12T09:56:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde20dc3700 1 Processor -- start 2023-12-12T09:56:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde20dc3700 1 -- start start 2023-12-12T09:56:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde20dc3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 0x7fde1c1973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:23.995 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde20dc3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c104830 0x7fde1c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde20dc3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde1c107160 0x7fde1c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde20dc3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c19ef00 con 0x7fde1c101f00 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde20dc3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c19f040 con 0x7fde1c107160 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde20dc3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c19f1d0 con 0x7fde1c104830 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde1a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 0x7fde1c1973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde1a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 0x7fde1c1973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33226/0 (socket says 172.21.15.42:33226) 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde1a59c700 1 -- 172.21.15.42:0/1381752589 learned_addr learned my addr 172.21.15.42:0/1381752589 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde19d9b700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c104830 0x7fde1c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde1ad9d700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde1c107160 0x7fde1c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde1a59c700 1 -- 172.21.15.42:0/1381752589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c104830 msgr2=0x7fde1c197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.991+0000 7fde1a59c700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c104830 0x7fde1c197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.992+0000 7fde1a59c700 1 -- 172.21.15.42:0/1381752589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde1c107160 msgr2=0x7fde1c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.992+0000 7fde1a59c700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde1c107160 0x7fde1c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.992+0000 7fde1a59c700 1 -- 172.21.15.42:0/1381752589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0c0095f0 con 0x7fde1c101f00 2023-12-12T09:56:23.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.992+0000 7fde1a59c700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 0x7fde1c1973d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0095c0 tx=0x7fde0c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.992+0000 7fde0b7fe700 1 -- 172.21.15.42:0/1381752589 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c021070 con 0x7fde1c101f00 2023-12-12T09:56:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.992+0000 7fde0b7fe700 1 -- 172.21.15.42:0/1381752589 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde0c0054f0 con 0x7fde1c101f00 2023-12-12T09:56:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.992+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c0ffcf0 con 0x7fde1c101f00 2023-12-12T09:56:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.992+0000 7fde0b7fe700 1 -- 172.21.15.42:0/1381752589 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c004ce0 con 0x7fde1c101f00 2023-12-12T09:56:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.992+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c100140 con 0x7fde1c101f00 2023-12-12T09:56:23.997 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.993+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c0fd470 con 0x7fde1c101f00 2023-12-12T09:56:23.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:23 smithi111 ceph-mon[102565]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:23.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:23 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:23.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:23 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:23.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:23 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:23.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:23 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:23.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:23 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:23.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:23 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.995+0000 7fde0b7fe700 1 -- 172.21.15.42:0/1381752589 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fde0c025070 con 0x7fde1c101f00 2023-12-12T09:56:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.995+0000 7fde0b7fe700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fde040a1480 0x7fde040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.995+0000 7fde0b7fe700 1 -- 172.21.15.42:0/1381752589 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fde0c0d1d30 con 0x7fde1c101f00 2023-12-12T09:56:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.995+0000 7fde19d9b700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fde040a1480 0x7fde040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:23.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.996+0000 7fde19d9b700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fde040a1480 0x7fde040a3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fde10005950 tx=0x7fde1000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:24.001 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:23.999+0000 7fde0b7fe700 1 -- 172.21.15.42:0/1381752589 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fde0c09b8c0 con 0x7fde1c101f00 2023-12-12T09:56:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.249+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde1c10cc80 con 0x7fde1c101f00 2023-12-12T09:56:24.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.249+0000 7fde0b7fe700 1 -- 172.21.15.42:0/1381752589 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6880 (secure 0 0 0) 0x7fde0c09eee0 con 0x7fde1c101f00 2023-12-12T09:56:24.252 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:24.252 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2023-12-12T09:56:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fde040a1480 msgr2=0x7fde040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fde040a1480 0x7fde040a3940 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fde10005950 tx=0x7fde1000b410 comp rx=0 tx=0).stop 2023-12-12T09:56:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 msgr2=0x7fde1c1973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 0x7fde1c1973d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde0c0095c0 tx=0x7fde0c01ca40 comp rx=0 tx=0).stop 2023-12-12T09:56:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 shutdown_connections 2023-12-12T09:56:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fde040a1480 0x7fde040a3940 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fde1c107160 0x7fde1c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:24.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fde1c101f00 0x7fde1c1973d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 --2- 172.21.15.42:0/1381752589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fde1c104830 0x7fde1c197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 >> 172.21.15.42:0/1381752589 conn(0x7fde1c0fb8f0 msgr2=0x7fde1c102840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 shutdown_connections 2023-12-12T09:56:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:24.252+0000 7fde20dc3700 1 -- 172.21.15.42:0/1381752589 wait complete. 2023-12-12T09:56:24.255 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 9 2023-12-12T09:56:24.932 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 09:56:24 smithi173 systemd[1]: Starting Ceph mds.i for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:24.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:24 smithi173 ceph-mon[103308]: Deploying daemon mds.i on smithi173 2023-12-12T09:56:24.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:24 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] up:boot 2023-12-12T09:56:24.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:24 smithi173 ceph-mon[103308]: fsmap 8 up:standby 2023-12-12T09:56:24.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:24 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T09:56:24.932 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:24 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1381752589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:24.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:24 smithi111 ceph-mon[102565]: Deploying daemon mds.i on smithi173 2023-12-12T09:56:24.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:24 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] up:boot 2023-12-12T09:56:24.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:24 smithi111 ceph-mon[102565]: fsmap 8 up:standby 2023-12-12T09:56:24.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:24 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T09:56:24.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:24 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1381752589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:25.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:24 smithi042 ceph-mon[98894]: Deploying daemon mds.i on smithi173 2023-12-12T09:56:25.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:24 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] up:boot 2023-12-12T09:56:25.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:24 smithi042 ceph-mon[98894]: fsmap 8 up:standby 2023-12-12T09:56:25.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:24 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T09:56:25.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:24 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1381752589' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:25.211 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 09:56:24 smithi173 podman[140484]: 2023-12-12T09:56:25.573 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 09:56:25 smithi173 ceph-mds[140524]: -- 172.21.15.173:0/202069956 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555a89e15520 con 0x555a89deb000 2023-12-12T09:56:25.573 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 09:56:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-i[140500]: starting mds.i at 2023-12-12T09:56:25.573 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 09:56:25 smithi173 bash[140484]: 7f9ddbf27eecfbf6658af417c818a6bd57cb2d5089885656a0f6eeb89454d26c 2023-12-12T09:56:25.573 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 09:56:25 smithi173 systemd[1]: Started Ceph mds.i for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:25.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:25 smithi111 ceph-mon[102565]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:25.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:25.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:25.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:25.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:25.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:25.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:25 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:26.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:25 smithi173 ceph-mon[103308]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:26.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:26.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:26.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:26.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:26.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:26.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:25 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:26.108 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:25 smithi042 ceph-mon[98894]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:26.109 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:26.109 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:26.109 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:26.109 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:26.109 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:26.109 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:25 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:26.267 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:26.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.401+0000 7f75f41bb700 1 -- 172.21.15.42:0/582613763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101f10 msgr2=0x7f75ec1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:26.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.401+0000 7f75f41bb700 1 --2- 172.21.15.42:0/582613763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101f10 0x7f75ec1087c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75e400b0d0 tx=0x7f75e400b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.401+0000 7f75f41bb700 1 -- 172.21.15.42:0/582613763 shutdown_connections 2023-12-12T09:56:26.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.401+0000 7f75f41bb700 1 --2- 172.21.15.42:0/582613763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75ec100380 0x7f75ec1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.401+0000 7f75f41bb700 1 --2- 172.21.15.42:0/582613763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101f10 0x7f75ec1087c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.401+0000 7f75f41bb700 1 --2- 172.21.15.42:0/582613763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75ec101570 0x7f75ec1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.401+0000 7f75f41bb700 1 -- 172.21.15.42:0/582613763 >> 172.21.15.42:0/582613763 conn(0x7f75ec0fb8f0 msgr2=0x7f75ec0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:26.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.402+0000 7f75f41bb700 1 -- 172.21.15.42:0/582613763 shutdown_connections 2023-12-12T09:56:26.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.402+0000 7f75f41bb700 1 -- 172.21.15.42:0/582613763 wait complete. 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.402+0000 7f75f41bb700 1 Processor -- start 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.402+0000 7f75f41bb700 1 -- start start 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f41bb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75ec100380 0x7f75ec195300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f41bb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101570 0x7f75ec195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f41bb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75ec101f10 0x7f75ec19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f41bb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec19ce30 con 0x7f75ec101570 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f41bb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec19cf70 con 0x7f75ec101f10 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f41bb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec19d100 con 0x7f75ec100380 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1756700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101570 0x7f75ec195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:26.407 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1756700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101570 0x7f75ec195840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33240/0 (socket says 172.21.15.42:33240) 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1756700 1 -- 172.21.15.42:0/478862991 learned_addr learned my addr 172.21.15.42:0/478862991 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1f57700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75ec100380 0x7f75ec195300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f2758700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75ec101f10 0x7f75ec19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1756700 1 -- 172.21.15.42:0/478862991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75ec100380 msgr2=0x7f75ec195300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1756700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75ec100380 0x7f75ec195300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1756700 1 -- 172.21.15.42:0/478862991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75ec101f10 msgr2=0x7f75ec19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1756700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75ec101f10 0x7f75ec19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1756700 1 -- 172.21.15.42:0/478862991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4009d00 con 0x7f75ec101570 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f2758700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75ec101f10 0x7f75ec19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f1756700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101570 0x7f75ec195840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f75e800b530 tx=0x7f75e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75e2ffd700 1 -- 172.21.15.42:0/478862991 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8011830 con 0x7f75ec101570 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75e2ffd700 1 -- 172.21.15.42:0/478862991 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e8011e50 con 0x7f75ec101570 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec19d3e0 con 0x7f75ec101570 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.403+0000 7f75e2ffd700 1 -- 172.21.15.42:0/478862991 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800f2e0 con 0x7f75ec101570 2023-12-12T09:56:26.408 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.404+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec19d9c0 con 0x7f75ec101570 2023-12-12T09:56:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.404+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec107730 con 0x7f75ec101570 2023-12-12T09:56:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.405+0000 7f75e2ffd700 1 -- 172.21.15.42:0/478862991 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f75e8010380 con 0x7f75ec101570 2023-12-12T09:56:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.406+0000 7f75e2ffd700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a3710 0x7f75d80a5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.406+0000 7f75f1f57700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a3710 0x7f75d80a5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:26.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.406+0000 7f75e2ffd700 1 -- 172.21.15.42:0/478862991 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f75e80cc360 con 0x7f75ec101570 2023-12-12T09:56:26.410 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.407+0000 7f75f1f57700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a3710 0x7f75d80a5bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f75ec1013d0 tx=0x7f75dc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:26.412 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.411+0000 7f75e2ffd700 1 -- 172.21.15.42:0/478862991 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f75e8095e60 con 0x7f75ec101570 2023-12-12T09:56:26.661 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.660+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75ec068230 con 0x7f75ec101570 2023-12-12T09:56:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.662+0000 7f75e2ffd700 1 -- 172.21.15.42:0/478862991 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7676 (secure 0 0 0) 0x7f75e8099480 con 0x7f75ec101570 2023-12-12T09:56:26.664 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:26.665 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.664+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a3710 msgr2=0x7f75d80a5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.664+0000 7f75f41bb700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a3710 0x7f75d80a5bd0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f75ec1013d0 tx=0x7f75dc00a380 comp rx=0 tx=0).stop 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.664+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101570 msgr2=0x7f75ec195840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.664+0000 7f75f41bb700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101570 0x7f75ec195840 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f75e800b530 tx=0x7f75e800b840 comp rx=0 tx=0).stop 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.665+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 shutdown_connections 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.665+0000 7f75f41bb700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a3710 0x7f75d80a5bd0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.665+0000 7f75f41bb700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75ec101f10 0x7f75ec19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.665+0000 7f75f41bb700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75ec100380 0x7f75ec195300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.665+0000 7f75f41bb700 1 --2- 172.21.15.42:0/478862991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75ec101570 0x7f75ec195840 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.665+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 >> 172.21.15.42:0/478862991 conn(0x7f75ec0fb8f0 msgr2=0x7f75ec104870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.665+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 shutdown_connections 2023-12-12T09:56:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:26.665+0000 7f75f41bb700 1 -- 172.21.15.42:0/478862991 wait complete. 2023-12-12T09:56:26.668 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 10 2023-12-12T09:56:26.679 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:26 smithi042 ceph-mon[98894]: Deploying daemon mds.g on smithi042 2023-12-12T09:56:26.679 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:26 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] up:boot 2023-12-12T09:56:26.679 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:26 smithi042 ceph-mon[98894]: fsmap 9 up:standby 2023-12-12T09:56:26.680 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:26 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-12T09:56:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:26 smithi111 ceph-mon[102565]: Deploying daemon mds.g on smithi042 2023-12-12T09:56:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:26 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] up:boot 2023-12-12T09:56:26.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:26 smithi111 ceph-mon[102565]: fsmap 9 up:standby 2023-12-12T09:56:26.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:26 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-12T09:56:27.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:26 smithi173 ceph-mon[103308]: Deploying daemon mds.g on smithi042 2023-12-12T09:56:27.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:26 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] up:boot 2023-12-12T09:56:27.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:26 smithi173 ceph-mon[103308]: fsmap 9 up:standby 2023-12-12T09:56:27.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:26 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-12T09:56:27.349 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 09:56:27 smithi042 systemd[1]: Starting Ceph mds.g for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:27.610 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 09:56:27 smithi042 podman[151106]: 2023-12-12T09:56:27.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:27 smithi042 ceph-mon[98894]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:27.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:27 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/478862991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:27.903 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 09:56:27 smithi042 ceph-mds[151146]: -- 172.21.15.42:0/787470320 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561abdffb520 con 0x561abdfd0c00 2023-12-12T09:56:27.903 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 09:56:27 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-g[151122]: starting mds.g at 2023-12-12T09:56:27.903 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 09:56:27 smithi042 bash[151106]: b19dca8b6377c63422cfc008ef9ef97d7c666b36a8aead0e074501aa048d8d7a 2023-12-12T09:56:27.903 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 09:56:27 smithi042 systemd[1]: Started Ceph mds.g for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:27.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:27 smithi111 ceph-mon[102565]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:27.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:27 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/478862991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:27 smithi173 ceph-mon[103308]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:28.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:27 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/478862991' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:28.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:28.711 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.42:6838/229824980,v1:172.21.15.42:6839/229824980] up:boot 2023-12-12T09:56:28.711 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: fsmap 10 up:standby 2023-12-12T09:56:28.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-12T09:56:28.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:28.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:28.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:28.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:28.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:28.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:28.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: Deploying daemon mds.a on smithi042 2023-12-12T09:56:28.712 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:28 smithi042 ceph-mon[98894]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.819+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3145658797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48100380 msgr2=0x7f7b481007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.819+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3145658797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48100380 0x7f7b481007a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7b380099c0 tx=0x7f7b38009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.819+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3145658797 shutdown_connections 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.819+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3145658797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48101f10 0x7f7b481087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.819+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3145658797 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48100380 0x7f7b481007a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.819+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3145658797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101570 0x7f7b481019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.819+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3145658797 >> 172.21.15.42:0/3145658797 conn(0x7f7b480fb8f0 msgr2=0x7f7b480fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.819+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3145658797 shutdown_connections 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.819+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3145658797 wait complete. 2023-12-12T09:56:28.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.820+0000 7f7b4c8f0700 1 Processor -- start 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.820+0000 7f7b4c8f0700 1 -- start start 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b4c8f0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48100380 0x7f7b48196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b4c8f0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101570 0x7f7b48196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b4c8f0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48101f10 0x7f7b4819d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b4c8f0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4819de70 con 0x7f7b48101f10 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b4c8f0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4819dfb0 con 0x7f7b48100380 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b4c8f0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4819e140 con 0x7f7b48101570 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b46d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48101f10 0x7f7b4819d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b46d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48101f10 0x7f7b4819d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33268/0 (socket says 172.21.15.42:33268) 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b46d9d700 1 -- 172.21.15.42:0/3770040174 learned_addr learned my addr 172.21.15.42:0/3770040174 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b4659c700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48100380 0x7f7b48196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b3ffff700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101570 0x7f7b48196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:28.826 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b46d9d700 1 -- 172.21.15.42:0/3770040174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101570 msgr2=0x7f7b48196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b46d9d700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101570 0x7f7b48196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b46d9d700 1 -- 172.21.15.42:0/3770040174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48100380 msgr2=0x7f7b48196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b46d9d700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48100380 0x7f7b48196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b46d9d700 1 -- 172.21.15.42:0/3770040174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b380095f0 con 0x7f7b48101f10 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b46d9d700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48101f10 0x7f7b4819d8a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000ef30 tx=0x7f7b4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.821+0000 7f7b3f7fe700 1 -- 172.21.15.42:0/3770040174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b40018070 con 0x7f7b48101f10 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.822+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4819e3c0 con 0x7f7b48101f10 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.822+0000 7f7b3f7fe700 1 -- 172.21.15.42:0/3770040174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b4000f040 con 0x7f7b48101f10 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.822+0000 7f7b3f7fe700 1 -- 172.21.15.42:0/3770040174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4000cdb0 con 0x7f7b48101f10 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.822+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4819e8d0 con 0x7f7b48101f10 2023-12-12T09:56:28.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.822+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b48107850 con 0x7f7b48101f10 2023-12-12T09:56:28.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.824+0000 7f7b3f7fe700 1 -- 172.21.15.42:0/3770040174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f7b40003a00 con 0x7f7b48101f10 2023-12-12T09:56:28.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.824+0000 7f7b3f7fe700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b340a1500 0x7f7b340a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:28.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.824+0000 7f7b4659c700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b340a1500 0x7f7b340a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:28.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.825+0000 7f7b3f7fe700 1 -- 172.21.15.42:0/3770040174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b400cc350 con 0x7f7b48101f10 2023-12-12T09:56:28.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.825+0000 7f7b4659c700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b340a1500 0x7f7b340a39c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009990 tx=0x7f7b38000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:28.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:28.829+0000 7f7b3f7fe700 1 -- 172.21.15.42:0/3770040174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7b40095ee0 con 0x7f7b48101f10 2023-12-12T09:56:28.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.42:6838/229824980,v1:172.21.15.42:6839/229824980] up:boot 2023-12-12T09:56:28.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: fsmap 10 up:standby 2023-12-12T09:56:28.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-12T09:56:28.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:28.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:28.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:28.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:28.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:28.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:28.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: Deploying daemon mds.a on smithi042 2023-12-12T09:56:28.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:28 smithi111 ceph-mon[102565]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.42:6838/229824980,v1:172.21.15.42:6839/229824980] up:boot 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: fsmap 10 up:standby 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: Deploying daemon mds.a on smithi042 2023-12-12T09:56:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:28 smithi173 ceph-mon[103308]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:29.080 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.078+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b48068230 con 0x7f7b48101f10 2023-12-12T09:56:29.081 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.079+0000 7f7b3f7fe700 1 -- 172.21.15.42:0/3770040174 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8465 (secure 0 0 0) 0x7f7b40099500 con 0x7f7b48101f10 2023-12-12T09:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:29.081 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b340a1500 msgr2=0x7f7b340a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b340a1500 0x7f7b340a39c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009990 tx=0x7f7b38000bc0 comp rx=0 tx=0).stop 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48101f10 msgr2=0x7f7b4819d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48101f10 0x7f7b4819d8a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000ef30 tx=0x7f7b4000c3a0 comp rx=0 tx=0).stop 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 shutdown_connections 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b340a1500 0x7f7b340a39c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48100380 0x7f7b48196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48101f10 0x7f7b4819d8a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 --2- 172.21.15.42:0/3770040174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101570 0x7f7b48196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 >> 172.21.15.42:0/3770040174 conn(0x7f7b480fb8f0 msgr2=0x7f7b48104990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:29.083 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 shutdown_connections 2023-12-12T09:56:29.084 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:29.081+0000 7f7b4c8f0700 1 -- 172.21.15.42:0/3770040174 wait complete. 2023-12-12T09:56:29.084 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 11 2023-12-12T09:56:29.653 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 09:56:29 smithi042 systemd[1]: Starting Ceph mds.a for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:29.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:29 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3770040174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:30.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:29 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3770040174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:30.100 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 09:56:29 smithi042 podman[151490]: 2023-12-12T09:56:30.101 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:29 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3770040174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:30.403 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 09:56:30 smithi042 ceph-mds[151530]: -- 172.21.15.42:0/1331252557 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5562d2e67520 con 0x5562d2e3d000 2023-12-12T09:56:30.403 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 09:56:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-a[151506]: starting mds.a at 2023-12-12T09:56:30.403 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 09:56:30 smithi042 bash[151490]: 7e3ddada8faca6b7a32c9aca419cb5a3e3a3f65964550fd004bf226091681138 2023-12-12T09:56:30.403 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 09:56:30 smithi042 systemd[1]: Started Ceph mds.a for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:31.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 -- 172.21.15.42:0/1916907571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4101570 msgr2=0x7fd7b41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/1916907571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4101570 0x7fd7b41019d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009920 tx=0x7fd7a8009c30 comp rx=0 tx=0).stop 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 -- 172.21.15.42:0/1916907571 shutdown_connections 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/1916907571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7b4101f10 0x7fd7b41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/1916907571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7b4100380 0x7fd7b41007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/1916907571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4101570 0x7fd7b41019d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 -- 172.21.15.42:0/1916907571 >> 172.21.15.42:0/1916907571 conn(0x7fd7b40fb8f0 msgr2=0x7fd7b40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 -- 172.21.15.42:0/1916907571 shutdown_connections 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 -- 172.21.15.42:0/1916907571 wait complete. 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 Processor -- start 2023-12-12T09:56:31.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.240+0000 7fd7bbd18700 1 -- start start 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7bbd18700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4100380 0x7fd7b41042c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7bbd18700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7b4101570 0x7fd7b4102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7bbd18700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7b4101f10 0x7fd7b4102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7bbd18700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b4103630 con 0x7fd7b4101f10 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7bbd18700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41037a0 con 0x7fd7b4100380 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7bbd18700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41a1510 con 0x7fd7b4101570 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7ba2b5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7b4101f10 0x7fd7b4102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7ba2b5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7b4101f10 0x7fd7b4102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33294/0 (socket says 172.21.15.42:33294) 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7ba2b5700 1 -- 172.21.15.42:0/2630837171 learned_addr learned my addr 172.21.15.42:0/2630837171 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7b9ab4700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4100380 0x7fd7b41042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7b92b3700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7b4101570 0x7fd7b4102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:31.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7ba2b5700 1 -- 172.21.15.42:0/2630837171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7b4101570 msgr2=0x7fd7b4102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7ba2b5700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7b4101570 0x7fd7b4102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7ba2b5700 1 -- 172.21.15.42:0/2630837171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4100380 msgr2=0x7fd7b41042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7ba2b5700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4100380 0x7fd7b41042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7ba2b5700 1 -- 172.21.15.42:0/2630837171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b00095f0 con 0x7fd7b4101f10 2023-12-12T09:56:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7b9ab4700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4100380 0x7fd7b41042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:56:31.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7ba2b5700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7b4101f10 0x7fd7b4102e90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00ef30 tx=0x7fd7ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:31.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7a6ffd700 1 -- 172.21.15.42:0/2630837171 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac018070 con 0x7fd7b4101f10 2023-12-12T09:56:31.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7a6ffd700 1 -- 172.21.15.42:0/2630837171 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7ac00f040 con 0x7fd7b4101f10 2023-12-12T09:56:31.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a8009520 con 0x7fd7b4101f10 2023-12-12T09:56:31.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.241+0000 7fd7a6ffd700 1 -- 172.21.15.42:0/2630837171 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac00cca0 con 0x7fd7b4101f10 2023-12-12T09:56:31.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.242+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b41a1ac0 con 0x7fd7b4101f10 2023-12-12T09:56:31.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.242+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b4068230 con 0x7fd7b4101f10 2023-12-12T09:56:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.244+0000 7fd7a6ffd700 1 -- 172.21.15.42:0/2630837171 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd7ac003a00 con 0x7fd7b4101f10 2023-12-12T09:56:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.244+0000 7fd7a6ffd700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7a00a1480 0x7fd7a00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.245+0000 7fd7b9ab4700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7a00a1480 0x7fd7a00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.245+0000 7fd7a6ffd700 1 -- 172.21.15.42:0/2630837171 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd7ac0ccdc0 con 0x7fd7b4101f10 2023-12-12T09:56:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.245+0000 7fd7b9ab4700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7a00a1480 0x7fd7a00a3940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7b41013d0 tx=0x7fd7b0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:31.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.248+0000 7fd7a6ffd700 1 -- 172.21.15.42:0/2630837171 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd7ac0968b0 con 0x7fd7b4101f10 2023-12-12T09:56:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:31.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: Deploying daemon mds.b on smithi111 2023-12-12T09:56:31.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.42:6840/4221329195,v1:172.21.15.42:6841/4221329195] up:boot 2023-12-12T09:56:31.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: fsmap 11 up:standby 2023-12-12T09:56:31.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:31 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-12T09:56:31.495 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:31.495 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:31.495 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:31.495 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:31.496 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:31.496 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:31.496 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:31.496 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: Deploying daemon mds.b on smithi111 2023-12-12T09:56:31.496 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.42:6840/4221329195,v1:172.21.15.42:6841/4221329195] up:boot 2023-12-12T09:56:31.496 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: fsmap 11 up:standby 2023-12-12T09:56:31.496 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:31 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-12T09:56:31.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.501+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b410bc00 con 0x7fd7b4101f10 2023-12-12T09:56:31.504 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.501+0000 7fd7a6ffd700 1 -- 172.21.15.42:0/2630837171 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9257 (secure 0 0 0) 0x7fd7ac099ed0 con 0x7fd7b4101f10 2023-12-12T09:56:31.504 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2023-12-12T09:56:31.507 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7a00a1480 msgr2=0x7fd7a00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:31.507 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7a00a1480 0x7fd7a00a3940 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7b41013d0 tx=0x7fd7b0009310 comp rx=0 tx=0).stop 2023-12-12T09:56:31.507 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7b4101f10 msgr2=0x7fd7b4102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:31.507 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7b4101f10 0x7fd7b4102e90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac00ef30 tx=0x7fd7ac00c3a0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.507 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 shutdown_connections 2023-12-12T09:56:31.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd7a00a1480 0x7fd7a00a3940 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b4100380 0x7fd7b41042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7b4101f10 0x7fd7b4102e90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 --2- 172.21.15.42:0/2630837171 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7b4101570 0x7fd7b4102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:31.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 >> 172.21.15.42:0/2630837171 conn(0x7fd7b40fb8f0 msgr2=0x7fd7b4068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:31.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 shutdown_connections 2023-12-12T09:56:31.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:31.504+0000 7fd7bbd18700 1 -- 172.21.15.42:0/2630837171 wait complete. 2023-12-12T09:56:31.508 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 12 2023-12-12T09:56:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-12T09:56:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-12T09:56:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: Deploying daemon mds.b on smithi111 2023-12-12T09:56:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.42:6840/4221329195,v1:172.21.15.42:6841/4221329195] up:boot 2023-12-12T09:56:31.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: fsmap 11 up:standby 2023-12-12T09:56:31.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:31 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-12T09:56:32.075 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:56:31 smithi111 systemd[1]: Starting Ceph mds.b for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T09:56:32.380 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:32 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2630837171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:32.380 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:56:32 smithi111 podman[139633]: 2023-12-12T09:56:32.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:32 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2630837171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:32.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:32 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2630837171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:32.747 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:56:32 smithi111 ceph-mds[139673]: -- 172.21.15.111:0/439305414 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556b4e169520 con 0x556b4e13e800 2023-12-12T09:56:32.747 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:56:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: starting mds.b at 2023-12-12T09:56:32.747 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:56:32 smithi111 bash[139633]: 24b6f60d49787ccbe7c6b3d3f7336cd47450e84bb367830a3b728a0080bb6b49 2023-12-12T09:56:32.747 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:56:32 smithi111 systemd[1]: Started Ceph mds.b for 968ccaec-98d3-11ee-95a3-87774f69a715. 2023-12-12T09:56:33.520 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:33.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.651+0000 7fb261347700 1 -- 172.21.15.42:0/655851319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c1030c0 msgr2=0x7fb25c109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:33.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.651+0000 7fb261347700 1 --2- 172.21.15.42:0/655851319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c1030c0 0x7fb25c109970 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb25400b0a0 tx=0x7fb25400b3b0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.651+0000 7fb261347700 1 -- 172.21.15.42:0/655851319 shutdown_connections 2023-12-12T09:56:33.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.651+0000 7fb261347700 1 --2- 172.21.15.42:0/655851319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c1030c0 0x7fb25c109970 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.651+0000 7fb261347700 1 --2- 172.21.15.42:0/655851319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb25c0ff750 0x7fb25c0ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.651+0000 7fb261347700 1 --2- 172.21.15.42:0/655851319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb25c102720 0x7fb25c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.651+0000 7fb261347700 1 -- 172.21.15.42:0/655851319 >> 172.21.15.42:0/655851319 conn(0x7fb25c0fb0a0 msgr2=0x7fb25c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:33.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.651+0000 7fb261347700 1 -- 172.21.15.42:0/655851319 shutdown_connections 2023-12-12T09:56:33.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.651+0000 7fb261347700 1 -- 172.21.15.42:0/655851319 wait complete. 2023-12-12T09:56:33.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb261347700 1 Processor -- start 2023-12-12T09:56:33.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb261347700 1 -- start start 2023-12-12T09:56:33.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb261347700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb25c0ff750 0x7fb25c197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:33.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb261347700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c102720 0x7fb25c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb261347700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb25c1030c0 0x7fb25c19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb261347700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c19f080 con 0x7fb25c102720 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb261347700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c19f1c0 con 0x7fb25c1030c0 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb261347700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25c19f350 con 0x7fb25c0ff750 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb25a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c102720 0x7fb25c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb25a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c102720 0x7fb25c197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33306/0 (socket says 172.21.15.42:33306) 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb25a7fc700 1 -- 172.21.15.42:0/820689508 learned_addr learned my addr 172.21.15.42:0/820689508 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb25affd700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb25c0ff750 0x7fb25c197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.652+0000 7fb25b7fe700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb25c1030c0 0x7fb25c19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb25a7fc700 1 -- 172.21.15.42:0/820689508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb25c0ff750 msgr2=0x7fb25c197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb25a7fc700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb25c0ff750 0x7fb25c197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb25a7fc700 1 -- 172.21.15.42:0/820689508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb25c1030c0 msgr2=0x7fb25c19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:33.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb25a7fc700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb25c1030c0 0x7fb25c19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb25a7fc700 1 -- 172.21.15.42:0/820689508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb254009d00 con 0x7fb25c102720 2023-12-12T09:56:33.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb25a7fc700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c102720 0x7fb25c197a90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d950 tx=0x7fb24c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:33.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb253fff700 1 -- 172.21.15.42:0/820689508 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00f890 con 0x7fb25c102720 2023-12-12T09:56:33.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb25c19f630 con 0x7fb25c102720 2023-12-12T09:56:33.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb253fff700 1 -- 172.21.15.42:0/820689508 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb24c010460 con 0x7fb25c102720 2023-12-12T09:56:33.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb253fff700 1 -- 172.21.15.42:0/820689508 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c00e610 con 0x7fb25c102720 2023-12-12T09:56:33.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.653+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb25c19fb90 con 0x7fb25c102720 2023-12-12T09:56:33.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.654+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb25c108960 con 0x7fb25c102720 2023-12-12T09:56:33.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.655+0000 7fb253fff700 1 -- 172.21.15.42:0/820689508 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb24c0108d0 con 0x7fb25c102720 2023-12-12T09:56:33.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.656+0000 7fb253fff700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2480a1480 0x7fb2480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:33.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.656+0000 7fb253fff700 1 -- 172.21.15.42:0/820689508 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb24c0cc630 con 0x7fb25c102720 2023-12-12T09:56:33.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.656+0000 7fb25affd700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2480a1480 0x7fb2480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:33.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.656+0000 7fb25affd700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2480a1480 0x7fb2480a3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb25c068410 tx=0x7fb24400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:33.661 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.659+0000 7fb253fff700 1 -- 172.21.15.42:0/820689508 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb24c096130 con 0x7fb25c102720 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] up:boot 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: fsmap 12 up:standby 2023-12-12T09:56:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:33 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-12T09:56:33.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:33.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:56:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] up:boot 2023-12-12T09:56:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: fsmap 12 up:standby 2023-12-12T09:56:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:33 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-12T09:56:33.937 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.935+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb25c0679a0 con 0x7fb25c102720 2023-12-12T09:56:33.938 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.936+0000 7fb253fff700 1 -- 172.21.15.42:0/820689508 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10052 (secure 0 0 0) 0x7fb24c099750 con 0x7fb25c102720 2023-12-12T09:56:33.938 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:33.938 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-12-12T09:56:33.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2480a1480 msgr2=0x7fb2480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2480a1480 0x7fb2480a3940 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb25c068410 tx=0x7fb24400a380 comp rx=0 tx=0).stop 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c102720 msgr2=0x7fb25c197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c102720 0x7fb25c197a90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb24c00d950 tx=0x7fb24c00dc60 comp rx=0 tx=0).stop 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 shutdown_connections 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2480a1480 0x7fb2480a3940 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb25c1030c0 0x7fb25c19eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb25c0ff750 0x7fb25c197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 --2- 172.21.15.42:0/820689508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb25c102720 0x7fb25c197a90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 >> 172.21.15.42:0/820689508 conn(0x7fb25c0fb0a0 msgr2=0x7fb25c105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 shutdown_connections 2023-12-12T09:56:33.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:33.938+0000 7fb261347700 1 -- 172.21.15.42:0/820689508 wait complete. 2023-12-12T09:56:33.942 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2023-12-12T09:56:33.954 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] up:boot 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: fsmap 12 up:standby 2023-12-12T09:56:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:33 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-12T09:56:34.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.089+0000 7f75f4909700 1 -- 172.21.15.42:0/4144934427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 msgr2=0x7f75f01098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.089+0000 7f75f4909700 1 --2- 172.21.15.42:0/4144934427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 0x7f75f01098f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f75e80099c0 tx=0x7f75e8009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.090+0000 7f75f4909700 1 -- 172.21.15.42:0/4144934427 shutdown_connections 2023-12-12T09:56:34.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.090+0000 7f75f4909700 1 --2- 172.21.15.42:0/4144934427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 0x7f75f01098f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.090+0000 7f75f4909700 1 --2- 172.21.15.42:0/4144934427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01014b0 0x7f75f01018d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.090+0000 7f75f4909700 1 --2- 172.21.15.42:0/4144934427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f01026a0 0x7f75f0102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.090+0000 7f75f4909700 1 -- 172.21.15.42:0/4144934427 >> 172.21.15.42:0/4144934427 conn(0x7f75f00fca20 msgr2=0x7f75f00fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:34.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.090+0000 7f75f4909700 1 -- 172.21.15.42:0/4144934427 shutdown_connections 2023-12-12T09:56:34.093 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.090+0000 7f75f4909700 1 -- 172.21.15.42:0/4144934427 wait complete. 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.090+0000 7f75f4909700 1 Processor -- start 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.090+0000 7f75f4909700 1 -- start start 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75f4909700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01014b0 0x7f75f0075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75f4909700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f01026a0 0x7f75f00739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75f4909700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 0x7f75f0073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75f4909700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f00746a0 con 0x7f75f0103040 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75f4909700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f0074810 con 0x7f75f01026a0 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75f4909700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f01a1410 con 0x7f75f01014b0 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75eed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 0x7f75f0073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75eed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 0x7f75f0073f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33322/0 (socket says 172.21.15.42:33322) 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75ee59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01014b0 0x7f75f0075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75eed9d700 1 -- 172.21.15.42:0/706695815 learned_addr learned my addr 172.21.15.42:0/706695815 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75edd9b700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f01026a0 0x7f75f00739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75eed9d700 1 -- 172.21.15.42:0/706695815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01014b0 msgr2=0x7f75f0075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75eed9d700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01014b0 0x7f75f0075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75eed9d700 1 -- 172.21.15.42:0/706695815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f01026a0 msgr2=0x7f75f00739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75eed9d700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f01026a0 0x7f75f00739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75eed9d700 1 -- 172.21.15.42:0/706695815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e80095f0 con 0x7f75f0103040 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75eed9d700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 0x7f75f0073f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f75e8004730 tx=0x7f75e8005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75df7fe700 1 -- 172.21.15.42:0/706695815 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e801d070 con 0x7f75f0103040 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f01a1640 con 0x7f75f0103040 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75df7fe700 1 -- 172.21.15.42:0/706695815 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75e8004050 con 0x7f75f0103040 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75df7fe700 1 -- 172.21.15.42:0/706695815 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8022e60 con 0x7f75f0103040 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.091+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f01a1a00 con 0x7f75f0103040 2023-12-12T09:56:34.096 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.092+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f0108860 con 0x7f75f0103040 2023-12-12T09:56:34.097 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.094+0000 7f75df7fe700 1 -- 172.21.15.42:0/706695815 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f75e8022470 con 0x7f75f0103040 2023-12-12T09:56:34.097 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.094+0000 7f75df7fe700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a1480 0x7f75d80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.097 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.094+0000 7f75ee59c700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a1480 0x7f75d80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.097 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.094+0000 7f75df7fe700 1 -- 172.21.15.42:0/706695815 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f75e80cde50 con 0x7f75f0103040 2023-12-12T09:56:34.097 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.095+0000 7f75ee59c700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a1480 0x7f75d80a3940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009550 tx=0x7f75e00091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:34.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.098+0000 7f75df7fe700 1 -- 172.21.15.42:0/706695815 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f75e80979e0 con 0x7f75f0103040 2023-12-12T09:56:34.316 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.314+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75f010ccd0 con 0x7f75f0103040 2023-12-12T09:56:34.318 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.316+0000 7f75df7fe700 1 -- 172.21.15.42:0/706695815 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18411 (secure 0 0 0) 0x7f75e809b000 con 0x7f75f0103040 2023-12-12T09:56:34.318 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:34.318 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":79,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:55:56.982365+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:56:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a1480 msgr2=0x7f75d80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a1480 0x7f75d80a3940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009550 tx=0x7f75e00091b0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 msgr2=0x7f75f0073f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 0x7f75f0073f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f75e8004730 tx=0x7f75e8005b00 comp rx=0 tx=0).stop 2023-12-12T09:56:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 shutdown_connections 2023-12-12T09:56:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f75d80a1480 0x7f75d80a3940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75f0103040 0x7f75f0073f00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f75f01014b0 0x7f75f0075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.320 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 --2- 172.21.15.42:0/706695815 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f75f01026a0 0x7f75f00739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 >> 172.21.15.42:0/706695815 conn(0x7f75f00fca20 msgr2=0x7f75f01059a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 shutdown_connections 2023-12-12T09:56:34.321 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.318+0000 7f75f4909700 1 -- 172.21.15.42:0/706695815 wait complete. 2023-12-12T09:56:34.334 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-12T09:56:34.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:56:34.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.467+0000 7fe2a04f4700 1 -- 172.21.15.42:0/4025088612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 msgr2=0x7fe298101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.467+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/4025088612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 0x7fe298101b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe2880099c0 tx=0x7fe288009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.467+0000 7fe2a04f4700 1 -- 172.21.15.42:0/4025088612 shutdown_connections 2023-12-12T09:56:34.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.467+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/4025088612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe298102040 0x7fe2981088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.467+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/4025088612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2981004b0 0x7fe2981008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.467+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/4025088612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 0x7fe298101b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.467+0000 7fe2a04f4700 1 -- 172.21.15.42:0/4025088612 >> 172.21.15.42:0/4025088612 conn(0x7fe2980fba20 msgr2=0x7fe2980fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.467+0000 7fe2a04f4700 1 -- 172.21.15.42:0/4025088612 shutdown_connections 2023-12-12T09:56:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.467+0000 7fe2a04f4700 1 -- 172.21.15.42:0/4025088612 wait complete. 2023-12-12T09:56:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe2a04f4700 1 Processor -- start 2023-12-12T09:56:34.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe2a04f4700 1 -- start start 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe2a04f4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2981004b0 0x7fe298197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe2a04f4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 0x7fe2981979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe2a04f4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe298102040 0x7fe29819e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe2a04f4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29819ef90 con 0x7fe2981016a0 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe2a04f4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29819f0d0 con 0x7fe298102040 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe2a04f4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29819f260 con 0x7fe2981004b0 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29da8f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 0x7fe2981979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29da8f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 0x7fe2981979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33334/0 (socket says 172.21.15.42:33334) 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29da8f700 1 -- 172.21.15.42:0/3149403354 learned_addr learned my addr 172.21.15.42:0/3149403354 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29ea91700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe298102040 0x7fe29819e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29e290700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2981004b0 0x7fe298197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29da8f700 1 -- 172.21.15.42:0/3149403354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2981004b0 msgr2=0x7fe298197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29da8f700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2981004b0 0x7fe298197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29da8f700 1 -- 172.21.15.42:0/3149403354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe298102040 msgr2=0x7fe29819e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29da8f700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe298102040 0x7fe29819e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29da8f700 1 -- 172.21.15.42:0/3149403354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880095f0 con 0x7fe2981016a0 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.468+0000 7fe29ea91700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe298102040 0x7fe29819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.469+0000 7fe29da8f700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 0x7fe2981979a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c010 tx=0x7fe288005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.469+0000 7fe28f7fe700 1 -- 172.21.15.42:0/3149403354 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28801d070 con 0x7fe2981016a0 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.469+0000 7fe28f7fe700 1 -- 172.21.15.42:0/3149403354 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe288005050 con 0x7fe2981016a0 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.469+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29819f4e0 con 0x7fe2981016a0 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.469+0000 7fe28f7fe700 1 -- 172.21.15.42:0/3149403354 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288004b30 con 0x7fe2981016a0 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.469+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29819f900 con 0x7fe2981016a0 2023-12-12T09:56:34.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.470+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe298067a80 con 0x7fe2981016a0 2023-12-12T09:56:34.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.471+0000 7fe28f7fe700 1 -- 172.21.15.42:0/3149403354 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe288004c90 con 0x7fe2981016a0 2023-12-12T09:56:34.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.472+0000 7fe28f7fe700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe2840a1480 0x7fe2840a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.472+0000 7fe29e290700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe2840a1480 0x7fe2840a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.472+0000 7fe28f7fe700 1 -- 172.21.15.42:0/3149403354 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe2880cd6a0 con 0x7fe2981016a0 2023-12-12T09:56:34.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.472+0000 7fe29e290700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe2840a1480 0x7fe2840a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe298101500 tx=0x7fe29400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:34.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.476+0000 7fe28f7fe700 1 -- 172.21.15.42:0/3149403354 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe288097230 con 0x7fe2981016a0 2023-12-12T09:56:34.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:34 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/820689508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:34.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:34 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/706695815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:34.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.693+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe29819fbc0 con 0x7fe2981016a0 2023-12-12T09:56:34.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.693+0000 7fe28f7fe700 1 -- 172.21.15.42:0/3149403354 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe28809a850 con 0x7fe2981016a0 2023-12-12T09:56:34.696 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T09:56:34.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.695+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe2840a1480 msgr2=0x7fe2840a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.695+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe2840a1480 0x7fe2840a3940 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe298101500 tx=0x7fe29400a3b0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.695+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 msgr2=0x7fe2981979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.695+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 0x7fe2981979a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe28800c010 tx=0x7fe288005f10 comp rx=0 tx=0).stop 2023-12-12T09:56:34.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.696+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 shutdown_connections 2023-12-12T09:56:34.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.696+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe2840a1480 0x7fe2840a3940 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.696+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe298102040 0x7fe29819e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.696+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2981004b0 0x7fe298197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.696+0000 7fe2a04f4700 1 --2- 172.21.15.42:0/3149403354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2981016a0 0x7fe2981979a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.696+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 >> 172.21.15.42:0/3149403354 conn(0x7fe2980fba20 msgr2=0x7fe2980fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:34.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.696+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 shutdown_connections 2023-12-12T09:56:34.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.696+0000 7fe2a04f4700 1 -- 172.21.15.42:0/3149403354 wait complete. 2023-12-12T09:56:34.710 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-12T09:56:34.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:34 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/820689508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:34.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:34 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/706695815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:34.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.837+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/3981785448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 msgr2=0x7fb2e41087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.837+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/3981785448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 0x7fb2e41087c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00b0d0 tx=0x7fb2dc00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.837+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/3981785448 shutdown_connections 2023-12-12T09:56:34.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.837+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/3981785448 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 0x7fb2e41087c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.837+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/3981785448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2e4100380 0x7fb2e41007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.837+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/3981785448 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e4101570 0x7fb2e41019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.839 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.837+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/3981785448 >> 172.21.15.42:0/3981785448 conn(0x7fb2e40fb8f0 msgr2=0x7fb2e40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:34.840 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.837+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/3981785448 shutdown_connections 2023-12-12T09:56:34.840 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.837+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/3981785448 wait complete. 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e8c0a700 1 Processor -- start 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e8c0a700 1 -- start start 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e8c0a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2e4100380 0x7fb2e4069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e8c0a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e4101570 0x7fb2e419c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e8c0a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 0x7fb2e419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e8c0a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e419eff0 con 0x7fb2e4101f10 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e8c0a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e419f160 con 0x7fb2e4101570 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e8c0a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e419f320 con 0x7fb2e4100380 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e2d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 0x7fb2e419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e2d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 0x7fb2e419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33348/0 (socket says 172.21.15.42:33348) 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e2d9d700 1 -- 172.21.15.42:0/1797011491 learned_addr learned my addr 172.21.15.42:0/1797011491 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e1d9b700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e4101570 0x7fb2e419c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.842 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e259c700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2e4100380 0x7fb2e4069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e2d9d700 1 -- 172.21.15.42:0/1797011491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2e4100380 msgr2=0x7fb2e4069150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e2d9d700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2e4100380 0x7fb2e4069150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e2d9d700 1 -- 172.21.15.42:0/1797011491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e4101570 msgr2=0x7fb2e419c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e2d9d700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e4101570 0x7fb2e419c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e2d9d700 1 -- 172.21.15.42:0/1797011491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc009d00 con 0x7fb2e4101f10 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.838+0000 7fb2e2d9d700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 0x7fb2e419e800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc007b60 tx=0x7fb2dc008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.839+0000 7fb2d37fe700 1 -- 172.21.15.42:0/1797011491 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00e040 con 0x7fb2e4101f10 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.839+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e419f540 con 0x7fb2e4101f10 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.839+0000 7fb2d37fe700 1 -- 172.21.15.42:0/1797011491 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2dc00be30 con 0x7fb2e4101f10 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.839+0000 7fb2d37fe700 1 -- 172.21.15.42:0/1797011491 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc01bae0 con 0x7fb2e4101f10 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.839+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e419fa10 con 0x7fb2e4101f10 2023-12-12T09:56:34.843 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.840+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e4107730 con 0x7fb2e4101f10 2023-12-12T09:56:34.844 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.841+0000 7fb2d37fe700 1 -- 172.21.15.42:0/1797011491 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb2dc01d670 con 0x7fb2e4101f10 2023-12-12T09:56:34.844 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.841+0000 7fb2d37fe700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:34.844 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.841+0000 7fb2e259c700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:34.844 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.841+0000 7fb2d37fe700 1 -- 172.21.15.42:0/1797011491 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb2dc0ce6e0 con 0x7fb2e4101f10 2023-12-12T09:56:34.844 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.842+0000 7fb2e259c700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005950 tx=0x7fb2d40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:34.846 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:34.845+0000 7fb2d37fe700 1 -- 172.21.15.42:0/1797011491 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb2dc098390 con 0x7fb2e4101f10 2023-12-12T09:56:34.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:34 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/820689508' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:34.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:34 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/706695815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:35.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.062+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb2e4068230 con 0x7fb2e4101f10 2023-12-12T09:56:35.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.063+0000 7fb2d37fe700 1 -- 172.21.15.42:0/1797011491 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb2dc09b9b0 con 0x7fb2e4101f10 2023-12-12T09:56:35.065 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:35.065 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2023-12-12T09:56:35.065 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T09:56:35.065 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2023-12-12T09:56:35.065 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2023-12-12T09:56:35.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T09:56:35.065 INFO:teuthology.orchestra.run.smithi042.stdout:] 2023-12-12T09:56:35.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2cc0a1480 msgr2=0x7fb2cc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:35.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb2d4005950 tx=0x7fb2d40058e0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 msgr2=0x7fb2e419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:35.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 0x7fb2e419e800 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc007b60 tx=0x7fb2dc008f10 comp rx=0 tx=0).stop 2023-12-12T09:56:35.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 shutdown_connections 2023-12-12T09:56:35.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb2cc0a1480 0x7fb2cc0a3940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb2e4101f10 0x7fb2e419e800 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2e4100380 0x7fb2e4069150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 --2- 172.21.15.42:0/1797011491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2e4101570 0x7fb2e419c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 >> 172.21.15.42:0/1797011491 conn(0x7fb2e40fb8f0 msgr2=0x7fb2e4104870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:35.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 shutdown_connections 2023-12-12T09:56:35.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.065+0000 7fb2e8c0a700 1 -- 172.21.15.42:0/1797011491 wait complete. 2023-12-12T09:56:35.081 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-12T09:56:35.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-12T09:56:35.216 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.212+0000 7fdbd6b40700 1 -- 172.21.15.42:0/860531099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0101670 msgr2=0x7fdbd0101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.212+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/860531099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0101670 0x7fdbd0101ad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0099c0 tx=0x7fdbbc009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 -- 172.21.15.42:0/860531099 shutdown_connections 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/860531099 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0102010 0x7fdbd01088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/860531099 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd0100480 0x7fdbd01008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/860531099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0101670 0x7fdbd0101ad0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 -- 172.21.15.42:0/860531099 >> 172.21.15.42:0/860531099 conn(0x7fdbd00fb9f0 msgr2=0x7fdbd00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 -- 172.21.15.42:0/860531099 shutdown_connections 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 -- 172.21.15.42:0/860531099 wait complete. 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 Processor -- start 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 -- start start 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd0100480 0x7fdbd0196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0101670 0x7fdbd0196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:35.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0102010 0x7fdbd019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd019df60 con 0x7fdbd0102010 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd019e0a0 con 0x7fdbd0101670 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.213+0000 7fdbd6b40700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd019e230 con 0x7fdbd0100480 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd50dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0102010 0x7fdbd019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd50dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0102010 0x7fdbd019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33362/0 (socket says 172.21.15.42:33362) 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd50dd700 1 -- 172.21.15.42:0/3785343270 learned_addr learned my addr 172.21.15.42:0/3785343270 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd48dc700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd0100480 0x7fdbd0196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbcffff700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0101670 0x7fdbd0196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd50dd700 1 -- 172.21.15.42:0/3785343270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd0100480 msgr2=0x7fdbd0196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd50dd700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd0100480 0x7fdbd0196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd50dd700 1 -- 172.21.15.42:0/3785343270 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0101670 msgr2=0x7fdbd0196970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd50dd700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0101670 0x7fdbd0196970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd50dd700 1 -- 172.21.15.42:0/3785343270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc0095f0 con 0x7fdbd0102010 2023-12-12T09:56:35.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd50dd700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0102010 0x7fdbd019d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800ef30 tx=0x7fdbc800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:35.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbcdffb700 1 -- 172.21.15.42:0/3785343270 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc8018070 con 0x7fdbd0102010 2023-12-12T09:56:35.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbcdffb700 1 -- 172.21.15.42:0/3785343270 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdbc800f040 con 0x7fdbd0102010 2023-12-12T09:56:35.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbcdffb700 1 -- 172.21.15.42:0/3785343270 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc800cd00 con 0x7fdbd0102010 2023-12-12T09:56:35.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd019e4b0 con 0x7fdbd0102010 2023-12-12T09:56:35.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.214+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd019e9c0 con 0x7fdbd0102010 2023-12-12T09:56:35.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.215+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd0107900 con 0x7fdbd0102010 2023-12-12T09:56:35.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.216+0000 7fdbcdffb700 1 -- 172.21.15.42:0/3785343270 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fdbc8003a00 con 0x7fdbd0102010 2023-12-12T09:56:35.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.217+0000 7fdbcdffb700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdbc00a1500 0x7fdbc00a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:35.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.217+0000 7fdbd48dc700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdbc00a1500 0x7fdbc00a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:35.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.217+0000 7fdbcdffb700 1 -- 172.21.15.42:0/3785343270 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdbc80cc300 con 0x7fdbd0102010 2023-12-12T09:56:35.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.217+0000 7fdbd48dc700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdbc00a1500 0x7fdbc00a39c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbd01014d0 tx=0x7fdbc4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:35.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.221+0000 7fdbcdffb700 1 -- 172.21.15.42:0/3785343270 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdbc8095e90 con 0x7fdbd0102010 2023-12-12T09:56:35.452 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:35.450+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdbd010bf20 con 0x7fdbd0102010 2023-12-12T09:56:36.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:35 smithi042 ceph-mon[98894]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:36.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:35 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3149403354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:36.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:35 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1797011491' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-12T09:56:36.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:35 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:35 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3785343270' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-12T09:56:36.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:35 smithi111 ceph-mon[102565]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:36.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:35 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3149403354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:36.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:35 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1797011491' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-12T09:56:36.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:35 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:35 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3785343270' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-12T09:56:36.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:35 smithi173 ceph-mon[103308]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:36.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:35 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3149403354' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:36.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:35 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1797011491' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-12T09:56:36.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:35 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:36.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:35 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3785343270' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-12T09:56:36.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.452+0000 7fdbcdffb700 1 -- 172.21.15.42:0/3785343270 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdbc80994b0 con 0x7fdbd0102010 2023-12-12T09:56:36.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdbc00a1500 msgr2=0x7fdbc00a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:36.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdbc00a1500 0x7fdbc00a39c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdbd01014d0 tx=0x7fdbc4009260 comp rx=0 tx=0).stop 2023-12-12T09:56:36.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0102010 msgr2=0x7fdbd019d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:36.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0102010 0x7fdbd019d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800ef30 tx=0x7fdbc800c3a0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 shutdown_connections 2023-12-12T09:56:36.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdbc00a1500 0x7fdbc00a39c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbd0102010 0x7fdbd019d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdbd0100480 0x7fdbd0196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 --2- 172.21.15.42:0/3785343270 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd0101670 0x7fdbd0196970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.455+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 >> 172.21.15.42:0/3785343270 conn(0x7fdbd00fb9f0 msgr2=0x7fdbd0104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:36.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.458+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 shutdown_connections 2023-12-12T09:56:36.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.459+0000 7fdbd6b40700 1 -- 172.21.15.42:0/3785343270 wait complete. 2023-12-12T09:56:36.462 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2023-12-12T09:56:36.474 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-12-12T09:56:36.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.607+0000 7f972fca5700 1 -- 172.21.15.42:0/1297587063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 msgr2=0x7f97281088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.607+0000 7f972fca5700 1 --2- 172.21.15.42:0/1297587063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 0x7f97281088c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f972000b0d0 tx=0x7f972000b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 -- 172.21.15.42:0/1297587063 shutdown_connections 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 --2- 172.21.15.42:0/1297587063 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728100480 0x7f97281008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 --2- 172.21.15.42:0/1297587063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 0x7f97281088c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 --2- 172.21.15.42:0/1297587063 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9728101670 0x7f9728101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 -- 172.21.15.42:0/1297587063 >> 172.21.15.42:0/1297587063 conn(0x7f97280fb9f0 msgr2=0x7f97280fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 -- 172.21.15.42:0/1297587063 shutdown_connections 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 -- 172.21.15.42:0/1297587063 wait complete. 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 Processor -- start 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 -- start start 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9728100480 0x7f9728197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728101670 0x7f9728197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 0x7f972819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:36.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972819f160 con 0x7f9728102010 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972819f2a0 con 0x7f9728101670 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.608+0000 7f972fca5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972819f430 con 0x7f9728100480 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972e242700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 0x7f972819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972e242700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 0x7f972819eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33376/0 (socket says 172.21.15.42:33376) 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972e242700 1 -- 172.21.15.42:0/3323028903 learned_addr learned my addr 172.21.15.42:0/3323028903 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972da41700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9728100480 0x7f9728197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972d240700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728101670 0x7f9728197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972e242700 1 -- 172.21.15.42:0/3323028903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9728100480 msgr2=0x7f9728197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972e242700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9728100480 0x7f9728197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972e242700 1 -- 172.21.15.42:0/3323028903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728101670 msgr2=0x7f9728197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972e242700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728101670 0x7f9728197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972e242700 1 -- 172.21.15.42:0/3323028903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9720009d00 con 0x7f9728102010 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972e242700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 0x7f972819eb90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9720007b60 tx=0x7f9720008e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f971affd700 1 -- 172.21.15.42:0/3323028903 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972000e040 con 0x7f9728102010 2023-12-12T09:56:36.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972819f6b0 con 0x7f9728102010 2023-12-12T09:56:36.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f971affd700 1 -- 172.21.15.42:0/3323028903 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f972001d930 con 0x7f9728102010 2023-12-12T09:56:36.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f971affd700 1 -- 172.21.15.42:0/3323028903 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972001b9f0 con 0x7f9728102010 2023-12-12T09:56:36.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.609+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972819fb30 con 0x7f9728102010 2023-12-12T09:56:36.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.610+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9728107830 con 0x7f9728102010 2023-12-12T09:56:36.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.612+0000 7f971affd700 1 -- 172.21.15.42:0/3323028903 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f972001daa0 con 0x7f9728102010 2023-12-12T09:56:36.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.613+0000 7f971affd700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97140a1480 0x7f97140a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:36.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.613+0000 7f972da41700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97140a1480 0x7f97140a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:36.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.613+0000 7f971affd700 1 -- 172.21.15.42:0/3323028903 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f97200ce810 con 0x7f9728102010 2023-12-12T09:56:36.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.614+0000 7f972da41700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97140a1480 0x7f97140a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97281014d0 tx=0x7f971c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.617+0000 7f971affd700 1 -- 172.21.15.42:0/3323028903 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9720098230 con 0x7f9728102010 2023-12-12T09:56:36.829 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:36.827+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f972810cff0 con 0x7f9728102010 2023-12-12T09:56:37.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.454+0000 7f971affd700 1 -- 172.21.15.42:0/3323028903 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v81) v1 ==== 171+0+0 (secure 0 0 0) 0x7f972009b850 con 0x7f9728102010 2023-12-12T09:56:37.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97140a1480 msgr2=0x7f97140a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:37.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97140a1480 0x7f97140a3940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97281014d0 tx=0x7f971c009260 comp rx=0 tx=0).stop 2023-12-12T09:56:37.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 msgr2=0x7f972819eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:37.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 0x7f972819eb90 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9720007b60 tx=0x7f9720008e90 comp rx=0 tx=0).stop 2023-12-12T09:56:37.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 shutdown_connections 2023-12-12T09:56:37.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97140a1480 0x7f97140a3940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9728102010 0x7f972819eb90 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9728100480 0x7f9728197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 --2- 172.21.15.42:0/3323028903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9728101670 0x7f9728197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.461+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 >> 172.21.15.42:0/3323028903 conn(0x7f97280fb9f0 msgr2=0x7f9728104970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:37.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.465+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 shutdown_connections 2023-12-12T09:56:37.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.471+0000 7f972fca5700 1 -- 172.21.15.42:0/3323028903 wait complete. 2023-12-12T09:56:37.480 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2023-12-12T09:56:37.495 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-12T09:56:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:56:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:37.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:56:37.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3785343270' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-12T09:56:37.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: osdmap e80: 12 total, 12 up, 12 in 2023-12-12T09:56:37.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:37 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3323028903' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.626+0000 7fed2959b700 1 -- 172.21.15.42:0/2691823360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24100480 msgr2=0x7fed241008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.626+0000 7fed2959b700 1 --2- 172.21.15.42:0/2691823360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24100480 0x7fed241008a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fed140099c0 tx=0x7fed14009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.626+0000 7fed2959b700 1 -- 172.21.15.42:0/2691823360 shutdown_connections 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.626+0000 7fed2959b700 1 --2- 172.21.15.42:0/2691823360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed24102010 0x7fed241088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.626+0000 7fed2959b700 1 --2- 172.21.15.42:0/2691823360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24100480 0x7fed241008a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.626+0000 7fed2959b700 1 --2- 172.21.15.42:0/2691823360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed24101670 0x7fed24101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.626+0000 7fed2959b700 1 -- 172.21.15.42:0/2691823360 >> 172.21.15.42:0/2691823360 conn(0x7fed240fb9f0 msgr2=0x7fed240fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.626+0000 7fed2959b700 1 -- 172.21.15.42:0/2691823360 shutdown_connections 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.626+0000 7fed2959b700 1 -- 172.21.15.42:0/2691823360 wait complete. 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed2959b700 1 Processor -- start 2023-12-12T09:56:37.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed2959b700 1 -- start start 2023-12-12T09:56:37.631 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed2959b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed24100480 0x7fed24069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed2959b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24101670 0x7fed2419c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed2959b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed24102010 0x7fed2419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed2959b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2419f060 con 0x7fed24101670 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed2959b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2419f1d0 con 0x7fed24102010 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed2959b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2419f390 con 0x7fed24100480 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed227fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24101670 0x7fed2419c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed227fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24101670 0x7fed2419c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33388/0 (socket says 172.21.15.42:33388) 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed227fc700 1 -- 172.21.15.42:0/4060541729 learned_addr learned my addr 172.21.15.42:0/4060541729 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed237fe700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed24102010 0x7fed2419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.627+0000 7fed22ffd700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed24100480 0x7fed24069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed227fc700 1 -- 172.21.15.42:0/4060541729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed24100480 msgr2=0x7fed24069150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed227fc700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed24100480 0x7fed24069150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed227fc700 1 -- 172.21.15.42:0/4060541729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed24102010 msgr2=0x7fed2419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:37.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed227fc700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed24102010 0x7fed2419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:37.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed227fc700 1 -- 172.21.15.42:0/4060541729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed140095f0 con 0x7fed24101670 2023-12-12T09:56:37.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed237fe700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed24102010 0x7fed2419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:56:37.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed227fc700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24101670 0x7fed2419c250 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fed1800d950 tx=0x7fed1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:37.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed0bfff700 1 -- 172.21.15.42:0/4060541729 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800f890 con 0x7fed24101670 2023-12-12T09:56:37.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2419f610 con 0x7fed24101670 2023-12-12T09:56:37.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed0bfff700 1 -- 172.21.15.42:0/4060541729 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed18010460 con 0x7fed24101670 2023-12-12T09:56:37.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed0bfff700 1 -- 172.21.15.42:0/4060541729 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1800e610 con 0x7fed24101670 2023-12-12T09:56:37.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.628+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2419fb20 con 0x7fed24101670 2023-12-12T09:56:37.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.629+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed24107900 con 0x7fed24101670 2023-12-12T09:56:37.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.630+0000 7fed0bfff700 1 -- 172.21.15.42:0/4060541729 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fed180108d0 con 0x7fed24101670 2023-12-12T09:56:37.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.631+0000 7fed0bfff700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fed0c0a1480 0x7fed0c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:37.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.631+0000 7fed22ffd700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fed0c0a1480 0x7fed0c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:37.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.631+0000 7fed0bfff700 1 -- 172.21.15.42:0/4060541729 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fed180cc9c0 con 0x7fed24101670 2023-12-12T09:56:37.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.631+0000 7fed22ffd700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fed0c0a1480 0x7fed0c0a3940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fed14009990 tx=0x7fed14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:37.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.634+0000 7fed0bfff700 1 -- 172.21.15.42:0/4060541729 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fed18096130 con 0x7fed24101670 2023-12-12T09:56:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:56:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:56:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3785343270' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-12T09:56:37.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: osdmap e80: 12 total, 12 up, 12 in 2023-12-12T09:56:37.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:37 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3323028903' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-12T09:56:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:56:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:37.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:56:37.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3785343270' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-12T09:56:37.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: osdmap e80: 12 total, 12 up, 12 in 2023-12-12T09:56:37.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:37 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3323028903' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-12T09:56:37.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:37.900+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fed24068310 con 0x7fed24101670 2023-12-12T09:56:38.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.458+0000 7fed0bfff700 1 -- 172.21.15.42:0/4060541729 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fed180160c0 con 0x7fed24101670 2023-12-12T09:56:38.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fed0c0a1480 msgr2=0x7fed0c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:38.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fed0c0a1480 0x7fed0c0a3940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fed14009990 tx=0x7fed14000bc0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24101670 msgr2=0x7fed2419c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:38.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24101670 0x7fed2419c250 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fed1800d950 tx=0x7fed1800dc60 comp rx=0 tx=0).stop 2023-12-12T09:56:38.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 shutdown_connections 2023-12-12T09:56:38.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fed0c0a1480 0x7fed0c0a3940 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed24102010 0x7fed2419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fed24100480 0x7fed24069150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 --2- 172.21.15.42:0/4060541729 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fed24101670 0x7fed2419c250 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 >> 172.21.15.42:0/4060541729 conn(0x7fed240fb9f0 msgr2=0x7fed24104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:38.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.462+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 shutdown_connections 2023-12-12T09:56:38.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.463+0000 7fed2959b700 1 -- 172.21.15.42:0/4060541729 wait complete. 2023-12-12T09:56:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 2 and data pool 3 2023-12-12T09:56:38.484 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-12-12T09:56:38.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-12-12T09:56:38.623 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.621+0000 7f2c33bc6700 1 -- 172.21.15.42:0/3602544943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 msgr2=0x7f2c2c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:38.623 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.621+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/3602544943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 0x7f2c2c1019c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c280099c0 tx=0x7f2c28009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.624 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.621+0000 7f2c33bc6700 1 -- 172.21.15.42:0/3602544943 shutdown_connections 2023-12-12T09:56:38.624 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.621+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/3602544943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2c103130 0x7f2c2c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.624 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.621+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/3602544943 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 0x7f2c2c1019c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.624 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.621+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/3602544943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c2c102790 0x7f2c2c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.624 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.621+0000 7f2c33bc6700 1 -- 172.21.15.42:0/3602544943 >> 172.21.15.42:0/3602544943 conn(0x7f2c2c0fcb10 msgr2=0x7f2c2c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:38.624 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.621+0000 7f2c33bc6700 1 -- 172.21.15.42:0/3602544943 shutdown_connections 2023-12-12T09:56:38.624 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.621+0000 7f2c33bc6700 1 -- 172.21.15.42:0/3602544943 wait complete. 2023-12-12T09:56:38.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.622+0000 7f2c33bc6700 1 Processor -- start 2023-12-12T09:56:38.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.622+0000 7f2c33bc6700 1 -- start start 2023-12-12T09:56:38.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.622+0000 7f2c33bc6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 0x7f2c2c197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:38.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.622+0000 7f2c33bc6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c2c102790 0x7f2c2c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:38.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.622+0000 7f2c33bc6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2c103130 0x7f2c2c19ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:38.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.622+0000 7f2c33bc6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c19f040 con 0x7f2c2c1015a0 2023-12-12T09:56:38.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.622+0000 7f2c33bc6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c19f180 con 0x7f2c2c103130 2023-12-12T09:56:38.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.622+0000 7f2c33bc6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c19f310 con 0x7f2c2c102790 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.622+0000 7f2c31962700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 0x7f2c2c197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31962700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 0x7f2c2c197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33408/0 (socket says 172.21.15.42:33408) 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31962700 1 -- 172.21.15.42:0/2589693418 learned_addr learned my addr 172.21.15.42:0/2589693418 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31161700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c2c102790 0x7f2c2c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c32163700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2c103130 0x7f2c2c19ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31962700 1 -- 172.21.15.42:0/2589693418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c2c102790 msgr2=0x7f2c2c197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31962700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c2c102790 0x7f2c2c197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31962700 1 -- 172.21.15.42:0/2589693418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2c103130 msgr2=0x7f2c2c19ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31962700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2c103130 0x7f2c2c19ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31962700 1 -- 172.21.15.42:0/2589693418 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c280095f0 con 0x7f2c2c1015a0 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31161700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c2c102790 0x7f2c2c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c31962700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 0x7f2c2c197510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c28003820 tx=0x7f2c28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c22ffd700 1 -- 172.21.15.42:0/2589693418 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2801d070 con 0x7f2c2c1015a0 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c073800 con 0x7f2c2c1015a0 2023-12-12T09:56:38.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c22ffd700 1 -- 172.21.15.42:0/2589693418 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c28004ca0 con 0x7f2c2c1015a0 2023-12-12T09:56:38.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c22ffd700 1 -- 172.21.15.42:0/2589693418 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28003d30 con 0x7f2c2c1015a0 2023-12-12T09:56:38.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.623+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c073d00 con 0x7f2c2c1015a0 2023-12-12T09:56:38.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.624+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c108a40 con 0x7f2c2c1015a0 2023-12-12T09:56:38.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.626+0000 7f2c22ffd700 1 -- 172.21.15.42:0/2589693418 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2c28005030 con 0x7f2c2c1015a0 2023-12-12T09:56:38.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.626+0000 7f2c22ffd700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2c180a1480 0x7f2c180a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:38.630 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.626+0000 7f2c22ffd700 1 -- 172.21.15.42:0/2589693418 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2c280cdd20 con 0x7f2c2c1015a0 2023-12-12T09:56:38.630 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.628+0000 7f2c31161700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2c180a1480 0x7f2c180a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:38.630 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.628+0000 7f2c31161700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2c180a1480 0x7f2c180a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c010420 tx=0x7f2c1c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:38.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.631+0000 7f2c22ffd700 1 -- 172.21.15.42:0/2589693418 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2c28097560 con 0x7f2c2c1015a0 2023-12-12T09:56:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:38 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3323028903' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-12T09:56:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:38 smithi111 ceph-mon[102565]: osdmap e81: 12 total, 12 up, 12 in 2023-12-12T09:56:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:38 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/4060541729' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-12T09:56:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:38 smithi111 ceph-mon[102565]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-12T09:56:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:38 smithi111 ceph-mon[102565]: osdmap e82: 12 total, 12 up, 12 in 2023-12-12T09:56:38.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:38 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3323028903' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-12T09:56:38.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:38 smithi173 ceph-mon[103308]: osdmap e81: 12 total, 12 up, 12 in 2023-12-12T09:56:38.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:38 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/4060541729' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-12T09:56:38.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:38 smithi173 ceph-mon[103308]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-12T09:56:38.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:38 smithi173 ceph-mon[103308]: osdmap e82: 12 total, 12 up, 12 in 2023-12-12T09:56:38.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:38.849+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f2c2c068760 con 0x7f2c2c1015a0 2023-12-12T09:56:38.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:38 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3323028903' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-12T09:56:38.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:38 smithi042 ceph-mon[98894]: osdmap e81: 12 total, 12 up, 12 in 2023-12-12T09:56:38.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:38 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/4060541729' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-12T09:56:38.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:38 smithi042 ceph-mon[98894]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-12T09:56:38.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:38 smithi042 ceph-mon[98894]: osdmap e82: 12 total, 12 up, 12 in 2023-12-12T09:56:39.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.260+0000 7f2c22ffd700 1 -- 172.21.15.42:0/2589693418 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2c2809ab80 con 0x7f2c2c1015a0 2023-12-12T09:56:39.264 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2c180a1480 msgr2=0x7f2c180a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2c180a1480 0x7f2c180a3940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c010420 tx=0x7f2c1c0073d0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 msgr2=0x7f2c2c197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 0x7f2c2c197510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c28003820 tx=0x7f2c28004730 comp rx=0 tx=0).stop 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 shutdown_connections 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2c180a1480 0x7f2c180a3940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2c2c103130 0x7f2c2c19ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c2c1015a0 0x7f2c2c197510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 --2- 172.21.15.42:0/2589693418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c2c102790 0x7f2c2c197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.262+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 >> 172.21.15.42:0/2589693418 conn(0x7f2c2c0fcb10 msgr2=0x7f2c2c105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.263+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 shutdown_connections 2023-12-12T09:56:39.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.263+0000 7f2c33bc6700 1 -- 172.21.15.42:0/2589693418 wait complete. 2023-12-12T09:56:39.277 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-12-12T09:56:39.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.411+0000 7fa212006700 1 -- 172.21.15.42:0/3238082026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 msgr2=0x7fa20c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.411+0000 7fa212006700 1 --2- 172.21.15.42:0/3238082026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 0x7fa20c101e30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009990 tx=0x7fa1fc009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.411+0000 7fa212006700 1 -- 172.21.15.42:0/3238082026 shutdown_connections 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.411+0000 7fa212006700 1 --2- 172.21.15.42:0/3238082026 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa20c100f90 0x7fa20c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.411+0000 7fa212006700 1 --2- 172.21.15.42:0/3238082026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 0x7fa20c101e30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.411+0000 7fa212006700 1 --2- 172.21.15.42:0/3238082026 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa20c1005d0 0x7fa20c100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.411+0000 7fa212006700 1 -- 172.21.15.42:0/3238082026 >> 172.21.15.42:0/3238082026 conn(0x7fa20c0fb080 msgr2=0x7fa20c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.411+0000 7fa212006700 1 -- 172.21.15.42:0/3238082026 shutdown_connections 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.411+0000 7fa212006700 1 -- 172.21.15.42:0/3238082026 wait complete. 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.412+0000 7fa212006700 1 Processor -- start 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.412+0000 7fa212006700 1 -- start start 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.412+0000 7fa212006700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa20c1005d0 0x7fa20c197430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.412+0000 7fa212006700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa20c100f90 0x7fa20c197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:39.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.412+0000 7fa212006700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 0x7fa20c19e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.412+0000 7fa212006700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c19ef60 con 0x7fa20c101a10 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.412+0000 7fa212006700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c19f0a0 con 0x7fa20c100f90 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.412+0000 7fa212006700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c19f230 con 0x7fa20c1005d0 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.412+0000 7fa20bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 0x7fa20c19e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 0x7fa20c19e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33426/0 (socket says 172.21.15.42:33426) 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20bfff700 1 -- 172.21.15.42:0/3175111364 learned_addr learned my addr 172.21.15.42:0/3175111364 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20b7fe700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa20c1005d0 0x7fa20c197430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20affd700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa20c100f90 0x7fa20c197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20bfff700 1 -- 172.21.15.42:0/3175111364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa20c1005d0 msgr2=0x7fa20c197430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20bfff700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa20c1005d0 0x7fa20c197430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20bfff700 1 -- 172.21.15.42:0/3175111364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa20c100f90 msgr2=0x7fa20c197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20bfff700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa20c100f90 0x7fa20c197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20bfff700 1 -- 172.21.15.42:0/3175111364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc0095f0 con 0x7fa20c101a10 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa20bfff700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 0x7fa20c19e990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa20400ef30 tx=0x7fa20400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa208ff9700 1 -- 172.21.15.42:0/3175111364 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa204018070 con 0x7fa20c101a10 2023-12-12T09:56:39.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa208ff9700 1 -- 172.21.15.42:0/3175111364 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa20400f040 con 0x7fa20c101a10 2023-12-12T09:56:39.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa20c19f4b0 con 0x7fa20c101a10 2023-12-12T09:56:39.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa208ff9700 1 -- 172.21.15.42:0/3175111364 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20400cd00 con 0x7fa20c101a10 2023-12-12T09:56:39.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.413+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c19f970 con 0x7fa20c101a10 2023-12-12T09:56:39.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.414+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c0679a0 con 0x7fa20c101a10 2023-12-12T09:56:39.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.415+0000 7fa208ff9700 1 -- 172.21.15.42:0/3175111364 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fa204003a00 con 0x7fa20c101a10 2023-12-12T09:56:39.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.416+0000 7fa208ff9700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1f40a1470 0x7fa1f40a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:39.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.416+0000 7fa208ff9700 1 -- 172.21.15.42:0/3175111364 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fa2040cc850 con 0x7fa20c101a10 2023-12-12T09:56:39.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.416+0000 7fa20b7fe700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1f40a1470 0x7fa1f40a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:39.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.416+0000 7fa20b7fe700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1f40a1470 0x7fa1f40a3930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00b2e0 tx=0x7fa1fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:39.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.420+0000 7fa208ff9700 1 -- 172.21.15.42:0/3175111364 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa204095f60 con 0x7fa20c101a10 2023-12-12T09:56:39.624 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:39.622+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa20c10bdd0 con 0x7fa20c101a10 2023-12-12T09:56:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: pgmap v256: 97 pgs: 15 creating+peering, 12 active+clean, 70 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/4060541729' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2589693418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2589693418' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-12T09:56:39.655 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:39 smithi042 ceph-mon[98894]: osdmap e83: 12 total, 12 up, 12 in 2023-12-12T09:56:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: pgmap v256: 97 pgs: 15 creating+peering, 12 active+clean, 70 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-12T09:56:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/4060541729' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-12T09:56:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-12T09:56:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-12T09:56:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2589693418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2589693418' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-12T09:56:39.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:39 smithi111 ceph-mon[102565]: osdmap e83: 12 total, 12 up, 12 in 2023-12-12T09:56:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: pgmap v256: 97 pgs: 15 creating+peering, 12 active+clean, 70 unknown; 577 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:56:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-12T09:56:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/4060541729' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-12T09:56:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-12T09:56:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-12T09:56:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2589693418' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-12T09:56:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2589693418' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-12T09:56:39.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:39 smithi173 ceph-mon[103308]: osdmap e83: 12 total, 12 up, 12 in 2023-12-12T09:56:40.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:40 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] up:active 2023-12-12T09:56:40.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:40 smithi111 ceph-mon[102565]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:56:40.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:40 smithi111 ceph-mon[102565]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:56:40.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:40 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3175111364' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-12T09:56:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:40 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] up:active 2023-12-12T09:56:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:40 smithi173 ceph-mon[103308]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:56:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:40 smithi173 ceph-mon[103308]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:56:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:40 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3175111364' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-12T09:56:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:40 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] up:active 2023-12-12T09:56:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:40 smithi042 ceph-mon[98894]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:56:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:40 smithi042 ceph-mon[98894]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:56:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:40 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3175111364' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-12T09:56:41.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.477+0000 7fa208ff9700 1 -- 172.21.15.42:0/3175111364 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) v1 ==== 228+0+0 (secure 0 0 0) 0x7fa204099580 con 0x7fa20c101a10 2023-12-12T09:56:41.482 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1f40a1470 msgr2=0x7fa1f40a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:41.482 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1f40a1470 0x7fa1f40a3930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00b2e0 tx=0x7fa1fc005fb0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.483 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 msgr2=0x7fa20c19e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:41.483 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 0x7fa20c19e990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa20400ef30 tx=0x7fa20400c3a0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.483 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 shutdown_connections 2023-12-12T09:56:41.483 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1f40a1470 0x7fa1f40a3930 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.483 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa20c101a10 0x7fa20c19e990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.483 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa20c1005d0 0x7fa20c197430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.483 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 --2- 172.21.15.42:0/3175111364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa20c100f90 0x7fa20c197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.483 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 >> 172.21.15.42:0/3175111364 conn(0x7fa20c0fb080 msgr2=0x7fa20c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:41.483 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.480+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 shutdown_connections 2023-12-12T09:56:41.488 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.485+0000 7fa212006700 1 -- 172.21.15.42:0/3175111364 wait complete. 2023-12-12T09:56:41.490 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data_ec' created 2023-12-12T09:56:41.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-12-12T09:56:41.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.634+0000 7faf247dd700 1 -- 172.21.15.42:0/1319961383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c1015a0 msgr2=0x7faf1c1019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:41.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.634+0000 7faf247dd700 1 --2- 172.21.15.42:0/1319961383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c1015a0 0x7faf1c1019c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7faf180099c0 tx=0x7faf18009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 -- 172.21.15.42:0/1319961383 shutdown_connections 2023-12-12T09:56:41.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 --2- 172.21.15.42:0/1319961383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf1c103130 0x7faf1c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 --2- 172.21.15.42:0/1319961383 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c1015a0 0x7faf1c1019c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 --2- 172.21.15.42:0/1319961383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf1c102790 0x7faf1c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 -- 172.21.15.42:0/1319961383 >> 172.21.15.42:0/1319961383 conn(0x7faf1c0fcb10 msgr2=0x7faf1c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:41.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 -- 172.21.15.42:0/1319961383 shutdown_connections 2023-12-12T09:56:41.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 -- 172.21.15.42:0/1319961383 wait complete. 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 Processor -- start 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 -- start start 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf1c1015a0 0x7faf1c1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c102790 0x7faf1c197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf1c103130 0x7faf1c19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c19f100 con 0x7faf1c102790 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c19f240 con 0x7faf1c1015a0 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.635+0000 7faf247dd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1c19f3d0 con 0x7faf1c103130 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf21d78700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c102790 0x7faf1c197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf21d78700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c102790 0x7faf1c197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33442/0 (socket says 172.21.15.42:33442) 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf21d78700 1 -- 172.21.15.42:0/1760723334 learned_addr learned my addr 172.21.15.42:0/1760723334 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:41.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf22579700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf1c1015a0 0x7faf1c1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf22d7a700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf1c103130 0x7faf1c19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf21d78700 1 -- 172.21.15.42:0/1760723334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf1c103130 msgr2=0x7faf1c19eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf21d78700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf1c103130 0x7faf1c19eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf21d78700 1 -- 172.21.15.42:0/1760723334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf1c1015a0 msgr2=0x7faf1c1975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf21d78700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf1c1015a0 0x7faf1c1975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf21d78700 1 -- 172.21.15.42:0/1760723334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf180095f0 con 0x7faf1c102790 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf21d78700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c102790 0x7faf1c197b10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00d950 tx=0x7faf0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf137fe700 1 -- 172.21.15.42:0/1760723334 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c009940 con 0x7faf1c102790 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1c19f6b0 con 0x7faf1c102790 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1c19fc10 con 0x7faf1c102790 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf137fe700 1 -- 172.21.15.42:0/1760723334 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf0c010460 con 0x7faf1c102790 2023-12-12T09:56:41.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.636+0000 7faf137fe700 1 -- 172.21.15.42:0/1760723334 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00f620 con 0x7faf1c102790 2023-12-12T09:56:41.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.638+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf1c108a70 con 0x7faf1c102790 2023-12-12T09:56:41.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.638+0000 7faf137fe700 1 -- 172.21.15.42:0/1760723334 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7faf0c009aa0 con 0x7faf1c102790 2023-12-12T09:56:41.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.639+0000 7faf137fe700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faf080a1480 0x7faf080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:41.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.639+0000 7faf22579700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faf080a1480 0x7faf080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:41.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.639+0000 7faf137fe700 1 -- 172.21.15.42:0/1760723334 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 8698+0+0 (secure 0 0 0) 0x7faf0c0ccd40 con 0x7faf1c102790 2023-12-12T09:56:41.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.639+0000 7faf22579700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faf080a1480 0x7faf080a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf18009f50 tx=0x7faf18005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:41.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.644+0000 7faf137fe700 1 -- 172.21.15.42:0/1760723334 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faf0c0961d0 con 0x7faf1c102790 2023-12-12T09:56:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:41 smithi111 ceph-mon[102565]: pgmap v259: 97 pgs: 15 creating+peering, 28 active+clean, 54 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-12-12T09:56:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:41 smithi111 ceph-mon[102565]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-12-12T09:56:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:41 smithi111 ceph-mon[102565]: Cluster is now healthy 2023-12-12T09:56:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:41 smithi111 ceph-mon[102565]: osdmap e84: 12 total, 12 up, 12 in 2023-12-12T09:56:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:41 smithi173 ceph-mon[103308]: pgmap v259: 97 pgs: 15 creating+peering, 28 active+clean, 54 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-12-12T09:56:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:41 smithi173 ceph-mon[103308]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-12-12T09:56:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:41 smithi173 ceph-mon[103308]: Cluster is now healthy 2023-12-12T09:56:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:41 smithi173 ceph-mon[103308]: osdmap e84: 12 total, 12 up, 12 in 2023-12-12T09:56:41.847 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:41.845+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7faf1c108450 con 0x7faf1c102790 2023-12-12T09:56:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:41 smithi042 ceph-mon[98894]: pgmap v259: 97 pgs: 15 creating+peering, 28 active+clean, 54 unknown; 578 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 2 op/s 2023-12-12T09:56:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:41 smithi042 ceph-mon[98894]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2023-12-12T09:56:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:41 smithi042 ceph-mon[98894]: Cluster is now healthy 2023-12-12T09:56:41.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:41 smithi042 ceph-mon[98894]: osdmap e84: 12 total, 12 up, 12 in 2023-12-12T09:56:42.485 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.483+0000 7faf137fe700 1 -- 172.21.15.42:0/1760723334 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7faf0c0160c0 con 0x7faf1c102790 2023-12-12T09:56:42.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faf080a1480 msgr2=0x7faf080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:42.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faf080a1480 0x7faf080a3940 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf18009f50 tx=0x7faf18005ea0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c102790 msgr2=0x7faf1c197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c102790 0x7faf1c197b10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00d950 tx=0x7faf0c00dc60 comp rx=0 tx=0).stop 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 shutdown_connections 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faf080a1480 0x7faf080a3940 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faf1c1015a0 0x7faf1c1975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faf1c103130 0x7faf1c19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 --2- 172.21.15.42:0/1760723334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf1c102790 0x7faf1c197b10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 >> 172.21.15.42:0/1760723334 conn(0x7faf1c0fcb10 msgr2=0x7faf1c105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 shutdown_connections 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.488+0000 7faf247dd700 1 -- 172.21.15.42:0/1760723334 wait complete. 2023-12-12T09:56:42.549 INFO:teuthology.orchestra.run.smithi042.stderr:set pool 4 allow_ec_overwrites to true 2023-12-12T09:56:42.550 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-12-12T09:56:42.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.684+0000 7ff863327700 1 -- 172.21.15.42:0/1875643991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c1016a0 msgr2=0x7ff85c101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:42.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.684+0000 7ff863327700 1 --2- 172.21.15.42:0/1875643991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c1016a0 0x7ff85c101b00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff8500099c0 tx=0x7ff850009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.684+0000 7ff863327700 1 -- 172.21.15.42:0/1875643991 shutdown_connections 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.684+0000 7ff863327700 1 --2- 172.21.15.42:0/1875643991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85c1004b0 0x7ff85c1008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.684+0000 7ff863327700 1 --2- 172.21.15.42:0/1875643991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff85c102040 0x7ff85c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.684+0000 7ff863327700 1 --2- 172.21.15.42:0/1875643991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c1016a0 0x7ff85c101b00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.684+0000 7ff863327700 1 -- 172.21.15.42:0/1875643991 >> 172.21.15.42:0/1875643991 conn(0x7ff85c0fba20 msgr2=0x7ff85c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.684+0000 7ff863327700 1 -- 172.21.15.42:0/1875643991 shutdown_connections 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.684+0000 7ff863327700 1 -- 172.21.15.42:0/1875643991 wait complete. 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff863327700 1 Processor -- start 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff863327700 1 -- start start 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff863327700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85c1004b0 0x7ff85c06a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff863327700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff85c1016a0 0x7ff85c19c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff863327700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c102040 0x7ff85c19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:42.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff863327700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c19eff0 con 0x7ff85c102040 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff863327700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c19f160 con 0x7ff85c1004b0 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff863327700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c19f320 con 0x7ff85c1016a0 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8618c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c102040 0x7ff85c19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8618c4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c102040 0x7ff85c19e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33466/0 (socket says 172.21.15.42:33466) 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8618c4700 1 -- 172.21.15.42:0/1832868174 learned_addr learned my addr 172.21.15.42:0/1832868174 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8608c2700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff85c1016a0 0x7ff85c19c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8610c3700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85c1004b0 0x7ff85c06a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8618c4700 1 -- 172.21.15.42:0/1832868174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff85c1016a0 msgr2=0x7ff85c19c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8618c4700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff85c1016a0 0x7ff85c19c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8618c4700 1 -- 172.21.15.42:0/1832868174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85c1004b0 msgr2=0x7ff85c06a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8618c4700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85c1004b0 0x7ff85c06a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:42.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8618c4700 1 -- 172.21.15.42:0/1832868174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8500095f0 con 0x7ff85c102040 2023-12-12T09:56:42.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8610c3700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85c1004b0 0x7ff85c06a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:56:42.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.685+0000 7ff8608c2700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff85c1016a0 0x7ff85c19c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:42.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.686+0000 7ff8618c4700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c102040 0x7ff85c19e850 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff854007ee0 tx=0x7ff85400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:42.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.686+0000 7ff84e7fc700 1 -- 172.21.15.42:0/1832868174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854017070 con 0x7ff85c102040 2023-12-12T09:56:42.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.686+0000 7ff84e7fc700 1 -- 172.21.15.42:0/1832868174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff85400f040 con 0x7ff85c102040 2023-12-12T09:56:42.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.686+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85c19f600 con 0x7ff85c102040 2023-12-12T09:56:42.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.686+0000 7ff84e7fc700 1 -- 172.21.15.42:0/1832868174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854005740 con 0x7ff85c102040 2023-12-12T09:56:42.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.686+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c19fa90 con 0x7ff85c102040 2023-12-12T09:56:42.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.687+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85c107900 con 0x7ff85c102040 2023-12-12T09:56:42.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.688+0000 7ff84e7fc700 1 -- 172.21.15.42:0/1832868174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff8540058a0 con 0x7ff85c102040 2023-12-12T09:56:42.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.689+0000 7ff84e7fc700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff8480a1480 0x7ff8480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:42.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.689+0000 7ff8610c3700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff8480a1480 0x7ff8480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:42.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.689+0000 7ff84e7fc700 1 -- 172.21.15.42:0/1832868174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 8698+0+0 (secure 0 0 0) 0x7ff8540cce10 con 0x7ff85c102040 2023-12-12T09:56:42.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.689+0000 7ff8610c3700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff8480a1480 0x7ff8480a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff858005fd0 tx=0x7ff858005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:42.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.693+0000 7ff84e7fc700 1 -- 172.21.15.42:0/1832868174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff8540962a0 con 0x7ff85c102040 2023-12-12T09:56:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:42 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3175111364' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-12T09:56:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:42 smithi111 ceph-mon[102565]: osdmap e85: 12 total, 12 up, 12 in 2023-12-12T09:56:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:42 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1760723334' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-12T09:56:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:42 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3175111364' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-12T09:56:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:42 smithi173 ceph-mon[103308]: osdmap e85: 12 total, 12 up, 12 in 2023-12-12T09:56:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:42 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1760723334' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-12T09:56:42.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:42 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3175111364' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-12T09:56:42.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:42 smithi042 ceph-mon[98894]: osdmap e85: 12 total, 12 up, 12 in 2023-12-12T09:56:42.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:42 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1760723334' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-12T09:56:42.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:42.947+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7ff85c067a80 con 0x7ff85c102040 2023-12-12T09:56:43.487 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.485+0000 7ff84e7fc700 1 -- 172.21.15.42:0/1832868174 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7ff8540998c0 con 0x7ff85c102040 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff8480a1480 msgr2=0x7ff8480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff8480a1480 0x7ff8480a3940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff858005fd0 tx=0x7ff858005f00 comp rx=0 tx=0).stop 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c102040 msgr2=0x7ff85c19e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c102040 0x7ff85c19e850 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff854007ee0 tx=0x7ff85400d3b0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 shutdown_connections 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff8480a1480 0x7ff8480a3940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff85c1004b0 0x7ff85c06a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff85c102040 0x7ff85c19e850 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 --2- 172.21.15.42:0/1832868174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff85c1016a0 0x7ff85c19c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 >> 172.21.15.42:0/1832868174 conn(0x7ff85c0fba20 msgr2=0x7ff85c104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:43.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 shutdown_connections 2023-12-12T09:56:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.489+0000 7ff863327700 1 -- 172.21.15.42:0/1832868174 wait complete. 2023-12-12T09:56:43.492 INFO:teuthology.orchestra.run.smithi042.stderr:added data pool 4 to fsmap 2023-12-12T09:56:43.497 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:56:43 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: 2023-12-12T09:56:43.243+0000 7fbe9c1df700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-12T09:56:43.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:43 smithi111 ceph-mon[102565]: pgmap v262: 129 pgs: 16 creating+peering, 83 active+clean, 30 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2023-12-12T09:56:43.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:43 smithi111 ceph-mon[102565]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-12T09:56:43.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:43 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1760723334' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-12T09:56:43.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:43 smithi111 ceph-mon[102565]: osdmap e86: 12 total, 12 up, 12 in 2023-12-12T09:56:43.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:43 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1832868174' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-12T09:56:43.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:43 smithi111 ceph-mon[102565]: osdmap e87: 12 total, 12 up, 12 in 2023-12-12T09:56:43.507 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.637+0000 7f2a6b190700 1 -- 172.21.15.42:0/3791728050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 msgr2=0x7f2a641098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.637+0000 7f2a6b190700 1 --2- 172.21.15.42:0/3791728050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 0x7f2a641098d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00b0d0 tx=0x7f2a5c00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.637+0000 7f2a6b190700 1 -- 172.21.15.42:0/3791728050 shutdown_connections 2023-12-12T09:56:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.637+0000 7f2a6b190700 1 --2- 172.21.15.42:0/3791728050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a64101490 0x7f2a641018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.637+0000 7f2a6b190700 1 --2- 172.21.15.42:0/3791728050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 0x7f2a641098d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.637+0000 7f2a6b190700 1 --2- 172.21.15.42:0/3791728050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a64102680 0x7f2a64102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.637+0000 7f2a6b190700 1 -- 172.21.15.42:0/3791728050 >> 172.21.15.42:0/3791728050 conn(0x7f2a640fca00 msgr2=0x7f2a640fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.637+0000 7f2a6b190700 1 -- 172.21.15.42:0/3791728050 shutdown_connections 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.637+0000 7f2a6b190700 1 -- 172.21.15.42:0/3791728050 wait complete. 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6b190700 1 Processor -- start 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6b190700 1 -- start start 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6b190700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a64101490 0x7f2a641974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6b190700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a64102680 0x7f2a64197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6b190700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 0x7f2a6419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6b190700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6419f020 con 0x7f2a64103020 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6b190700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6419f160 con 0x7f2a64101490 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6b190700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6419f2f0 con 0x7f2a64102680 2023-12-12T09:56:43.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6972d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 0x7f2a6419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6972d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 0x7f2a6419ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33480/0 (socket says 172.21.15.42:33480) 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6972d700 1 -- 172.21.15.42:0/1255282620 learned_addr learned my addr 172.21.15.42:0/1255282620 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a68f2c700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a64101490 0x7f2a641974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a63fff700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a64102680 0x7f2a64197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6972d700 1 -- 172.21.15.42:0/1255282620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a64102680 msgr2=0x7f2a64197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6972d700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a64102680 0x7f2a64197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6972d700 1 -- 172.21.15.42:0/1255282620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a64101490 msgr2=0x7f2a641974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6972d700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a64101490 0x7f2a641974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.638+0000 7f2a6972d700 1 -- 172.21.15.42:0/1255282620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c009d00 con 0x7f2a64103020 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.639+0000 7f2a6972d700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 0x7f2a6419ea50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c007bd0 tx=0x7f2a5c0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.639+0000 7f2a61ffb700 1 -- 172.21.15.42:0/1255282620 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00e040 con 0x7f2a64103020 2023-12-12T09:56:43.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.639+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6419f570 con 0x7f2a64103020 2023-12-12T09:56:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.639+0000 7f2a61ffb700 1 -- 172.21.15.42:0/1255282620 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a5c012dc0 con 0x7f2a64103020 2023-12-12T09:56:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.639+0000 7f2a61ffb700 1 -- 172.21.15.42:0/1255282620 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c01b450 con 0x7f2a64103020 2023-12-12T09:56:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.639+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6419f9a0 con 0x7f2a64103020 2023-12-12T09:56:43.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.640+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a64108840 con 0x7f2a64103020 2023-12-12T09:56:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.641+0000 7f2a61ffb700 1 -- 172.21.15.42:0/1255282620 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f2a5c01d610 con 0x7f2a64103020 2023-12-12T09:56:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.642+0000 7f2a61ffb700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a540a1480 0x7f2a540a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.642+0000 7f2a61ffb700 1 -- 172.21.15.42:0/1255282620 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2a5c0cf4b0 con 0x7f2a64103020 2023-12-12T09:56:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.642+0000 7f2a68f2c700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a540a1480 0x7f2a540a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:43.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.642+0000 7f2a68f2c700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a540a1480 0x7f2a540a3940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a58005950 tx=0x7f2a580058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:43.647 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.645+0000 7f2a61ffb700 1 -- 172.21.15.42:0/1255282620 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2a5c0602f0 con 0x7f2a64103020 2023-12-12T09:56:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:43 smithi173 ceph-mon[103308]: pgmap v262: 129 pgs: 16 creating+peering, 83 active+clean, 30 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2023-12-12T09:56:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:43 smithi173 ceph-mon[103308]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-12T09:56:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:43 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1760723334' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-12T09:56:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:43 smithi173 ceph-mon[103308]: osdmap e86: 12 total, 12 up, 12 in 2023-12-12T09:56:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:43 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1832868174' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-12T09:56:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:43 smithi173 ceph-mon[103308]: osdmap e87: 12 total, 12 up, 12 in 2023-12-12T09:56:43.899 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.897+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a64068680 con 0x7f2a64103020 2023-12-12T09:56:43.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.898+0000 7f2a61ffb700 1 -- 172.21.15.42:0/1255282620 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11270 (secure 0 0 0) 0x7f2a5c0989b0 con 0x7f2a64103020 2023-12-12T09:56:43.900 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:43.900 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T09:56:42.948706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a540a1480 msgr2=0x7f2a540a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a540a1480 0x7f2a540a3940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a58005950 tx=0x7f2a580058e0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 msgr2=0x7f2a6419ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 0x7f2a6419ea50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c007bd0 tx=0x7f2a5c0094f0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 shutdown_connections 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a540a1480 0x7f2a540a3940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a64101490 0x7f2a641974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2a64103020 0x7f2a6419ea50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 --2- 172.21.15.42:0/1255282620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a64102680 0x7f2a64197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 >> 172.21.15.42:0/1255282620 conn(0x7f2a640fca00 msgr2=0x7f2a64105980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:43.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 shutdown_connections 2023-12-12T09:56:43.903 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:43.900+0000 7f2a6b190700 1 -- 172.21.15.42:0/1255282620 wait complete. 2023-12-12T09:56:43.903 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2023-12-12T09:56:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:43 smithi042 ceph-mon[98894]: pgmap v262: 129 pgs: 16 creating+peering, 83 active+clean, 30 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 3 op/s 2023-12-12T09:56:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:43 smithi042 ceph-mon[98894]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-12-12T09:56:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:43 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1760723334' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-12T09:56:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:43 smithi042 ceph-mon[98894]: osdmap e86: 12 total, 12 up, 12 in 2023-12-12T09:56:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:43 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1832868174' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-12T09:56:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:43 smithi042 ceph-mon[98894]: osdmap e87: 12 total, 12 up, 12 in 2023-12-12T09:56:43.915 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:56:44.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.038+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/3738754479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78101f30 msgr2=0x7fcf78104320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.038+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/3738754479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78101f30 0x7fcf78104320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcf68009990 tx=0x7fcf68009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.039+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/3738754479 shutdown_connections 2023-12-12T09:56:44.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.039+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/3738754479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf78107190 0x7fcf78109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.039+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/3738754479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78101f30 0x7fcf78104320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.039+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/3738754479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf78104860 0x7fcf78106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.039+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/3738754479 >> 172.21.15.42:0/3738754479 conn(0x7fcf780fb920 msgr2=0x7fcf780fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:44.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.039+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/3738754479 shutdown_connections 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.039+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/3738754479 wait complete. 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.039+0000 7fcf7cd7f700 1 Processor -- start 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.039+0000 7fcf7cd7f700 1 -- start start 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf7cd7f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf78101f30 0x7fcf781974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf7cd7f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78104860 0x7fcf78197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf7cd7f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf78107190 0x7fcf7819ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf7cd7f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7819f000 con 0x7fcf78104860 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf7cd7f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7819f140 con 0x7fcf78107190 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf7cd7f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7819f2d0 con 0x7fcf78101f30 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf75d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78104860 0x7fcf78197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf75d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78104860 0x7fcf78197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33492/0 (socket says 172.21.15.42:33492) 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf75d9b700 1 -- 172.21.15.42:0/909848740 learned_addr learned my addr 172.21.15.42:0/909848740 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf7659c700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf78101f30 0x7fcf781974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf76d9d700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf78107190 0x7fcf7819ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf75d9b700 1 -- 172.21.15.42:0/909848740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf78101f30 msgr2=0x7fcf781974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf75d9b700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf78101f30 0x7fcf781974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf75d9b700 1 -- 172.21.15.42:0/909848740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf78107190 msgr2=0x7fcf7819ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf75d9b700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf78107190 0x7fcf7819ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf75d9b700 1 -- 172.21.15.42:0/909848740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf680095f0 con 0x7fcf78104860 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf75d9b700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78104860 0x7fcf78197a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00b530 tx=0x7fcf6c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf677fe700 1 -- 172.21.15.42:0/909848740 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c011830 con 0x7fcf78104860 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf677fe700 1 -- 172.21.15.42:0/909848740 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf6c011e50 con 0x7fcf78104860 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7819f5b0 con 0x7fcf78104860 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf677fe700 1 -- 172.21.15.42:0/909848740 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00f2e0 con 0x7fcf78104860 2023-12-12T09:56:44.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.040+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7819fb10 con 0x7fcf78104860 2023-12-12T09:56:44.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.042+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf780fd4a0 con 0x7fcf78104860 2023-12-12T09:56:44.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.042+0000 7fcf677fe700 1 -- 172.21.15.42:0/909848740 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fcf6c010580 con 0x7fcf78104860 2023-12-12T09:56:44.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.043+0000 7fcf677fe700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf600a1500 0x7fcf600a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.043+0000 7fcf677fe700 1 -- 172.21.15.42:0/909848740 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcf6c0cc9a0 con 0x7fcf78104860 2023-12-12T09:56:44.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.043+0000 7fcf7659c700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf600a1500 0x7fcf600a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.044+0000 7fcf7659c700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf600a1500 0x7fcf600a39c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006010 tx=0x7fcf680037b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:44.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.048+0000 7fcf677fe700 1 -- 172.21.15.42:0/909848740 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fcf6c095e10 con 0x7fcf78104860 2023-12-12T09:56:44.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.250+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf780fce80 con 0x7fcf78104860 2023-12-12T09:56:44.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.252+0000 7fcf677fe700 1 -- 172.21.15.42:0/909848740 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7fcf6c099430 con 0x7fcf78104860 2023-12-12T09:56:44.254 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:44.254 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:56:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf600a1500 msgr2=0x7fcf600a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf600a1500 0x7fcf600a39c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcf68006010 tx=0x7fcf680037b0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78104860 msgr2=0x7fcf78197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78104860 0x7fcf78197a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00b530 tx=0x7fcf6c00b8f0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 shutdown_connections 2023-12-12T09:56:44.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcf600a1500 0x7fcf600a39c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcf78107190 0x7fcf7819ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf78101f30 0x7fcf781974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 --2- 172.21.15.42:0/909848740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf78104860 0x7fcf78197a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 >> 172.21.15.42:0/909848740 conn(0x7fcf780fb920 msgr2=0x7fcf78102870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:44.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 shutdown_connections 2023-12-12T09:56:44.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.254+0000 7fcf7cd7f700 1 -- 172.21.15.42:0/909848740 wait complete. 2023-12-12T09:56:44.269 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:56:44.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.393+0000 7f83c8c79700 1 -- 172.21.15.42:0/1734167782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41015a0 msgr2=0x7f83c4101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.393+0000 7f83c8c79700 1 --2- 172.21.15.42:0/1734167782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41015a0 0x7f83c4101a00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f83b80099c0 tx=0x7f83b8009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.393+0000 7f83c8c79700 1 -- 172.21.15.42:0/1734167782 shutdown_connections 2023-12-12T09:56:44.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.393+0000 7f83c8c79700 1 --2- 172.21.15.42:0/1734167782 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c4101f40 0x7f83c41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.393+0000 7f83c8c79700 1 --2- 172.21.15.42:0/1734167782 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c41003b0 0x7f83c41007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.393+0000 7f83c8c79700 1 --2- 172.21.15.42:0/1734167782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41015a0 0x7f83c4101a00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.393+0000 7f83c8c79700 1 -- 172.21.15.42:0/1734167782 >> 172.21.15.42:0/1734167782 conn(0x7f83c40fb920 msgr2=0x7f83c40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:44.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.393+0000 7f83c8c79700 1 -- 172.21.15.42:0/1734167782 shutdown_connections 2023-12-12T09:56:44.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.393+0000 7f83c8c79700 1 -- 172.21.15.42:0/1734167782 wait complete. 2023-12-12T09:56:44.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c8c79700 1 Processor -- start 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c8c79700 1 -- start start 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c8c79700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41003b0 0x7f83c4197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c8c79700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c41015a0 0x7f83c4197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c8c79700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c4101f40 0x7f83c419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c8c79700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c419ef40 con 0x7f83c41003b0 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c8c79700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c419f080 con 0x7f83c4101f40 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c8c79700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c419f210 con 0x7f83c41015a0 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41003b0 0x7f83c4197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41003b0 0x7f83c4197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33504/0 (socket says 172.21.15.42:33504) 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c259c700 1 -- 172.21.15.42:0/840826137 learned_addr learned my addr 172.21.15.42:0/840826137 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c1d9b700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c41015a0 0x7f83c4197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c2d9d700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c4101f40 0x7f83c419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c259c700 1 -- 172.21.15.42:0/840826137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c41015a0 msgr2=0x7f83c4197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c259c700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c41015a0 0x7f83c4197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c259c700 1 -- 172.21.15.42:0/840826137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c4101f40 msgr2=0x7f83c419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c259c700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c4101f40 0x7f83c419e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c259c700 1 -- 172.21.15.42:0/840826137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b80095f0 con 0x7f83c41003b0 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.394+0000 7f83c259c700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41003b0 0x7f83c4197410 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f83b400d980 tx=0x7f83b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.395+0000 7f83b37fe700 1 -- 172.21.15.42:0/840826137 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4004020 con 0x7f83c41003b0 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.395+0000 7f83b37fe700 1 -- 172.21.15.42:0/840826137 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b400f690 con 0x7f83c41003b0 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.395+0000 7f83b37fe700 1 -- 172.21.15.42:0/840826137 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b4003a60 con 0x7f83c41003b0 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.395+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c419f4f0 con 0x7f83c41003b0 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.395+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c419fa00 con 0x7f83c41003b0 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.396+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c40679a0 con 0x7f83c41003b0 2023-12-12T09:56:44.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.397+0000 7f83b37fe700 1 -- 172.21.15.42:0/840826137 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f83b400fb00 con 0x7f83c41003b0 2023-12-12T09:56:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.397+0000 7f83b37fe700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83ac0a1500 0x7f83ac0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.398+0000 7f83c1d9b700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83ac0a1500 0x7f83ac0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.398+0000 7f83b37fe700 1 -- 172.21.15.42:0/840826137 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f83b40ccda0 con 0x7f83c41003b0 2023-12-12T09:56:44.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.398+0000 7f83c1d9b700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83ac0a1500 0x7f83ac0a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83b8006010 tx=0x7f83b80037c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:44.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.401+0000 7f83b37fe700 1 -- 172.21.15.42:0/840826137 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f83b4096290 con 0x7f83c41003b0 2023-12-12T09:56:44.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.607+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83c410bed0 con 0x7f83c41003b0 2023-12-12T09:56:44.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.608+0000 7f83b37fe700 1 -- 172.21.15.42:0/840826137 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7f83b40998b0 con 0x7f83c41003b0 2023-12-12T09:56:44.610 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:44.611 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:56:44.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83ac0a1500 msgr2=0x7f83ac0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83ac0a1500 0x7f83ac0a39c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83b8006010 tx=0x7f83b80037c0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41003b0 msgr2=0x7f83c4197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41003b0 0x7f83c4197410 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f83b400d980 tx=0x7f83b400dc90 comp rx=0 tx=0).stop 2023-12-12T09:56:44.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 shutdown_connections 2023-12-12T09:56:44.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83ac0a1500 0x7f83ac0a39c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83c4101f40 0x7f83c419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83c41003b0 0x7f83c4197410 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 --2- 172.21.15.42:0/840826137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83c41015a0 0x7f83c4197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 >> 172.21.15.42:0/840826137 conn(0x7f83c40fb920 msgr2=0x7f83c4069810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:44.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 shutdown_connections 2023-12-12T09:56:44.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.610+0000 7f83c8c79700 1 -- 172.21.15.42:0/840826137 wait complete. 2023-12-12T09:56:44.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:44 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1832868174' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-12T09:56:44.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:44 smithi042 ceph-mon[98894]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:56:44.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:44 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1255282620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:44.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:44 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/909848740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:44.643 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:56:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.767+0000 7ff4247b3700 1 -- 172.21.15.42:0/2880145951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c103030 msgr2=0x7ff41c1098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.767+0000 7ff4247b3700 1 --2- 172.21.15.42:0/2880145951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c103030 0x7ff41c1098e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b0d0 tx=0x7ff41400b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.767+0000 7ff4247b3700 1 -- 172.21.15.42:0/2880145951 shutdown_connections 2023-12-12T09:56:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.767+0000 7ff4247b3700 1 --2- 172.21.15.42:0/2880145951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff41c1014a0 0x7ff41c1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.767+0000 7ff4247b3700 1 --2- 172.21.15.42:0/2880145951 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c103030 0x7ff41c1098e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.767+0000 7ff4247b3700 1 --2- 172.21.15.42:0/2880145951 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff41c102690 0x7ff41c102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.767+0000 7ff4247b3700 1 -- 172.21.15.42:0/2880145951 >> 172.21.15.42:0/2880145951 conn(0x7ff41c0fca30 msgr2=0x7ff41c0fee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.767+0000 7ff4247b3700 1 -- 172.21.15.42:0/2880145951 shutdown_connections 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.767+0000 7ff4247b3700 1 -- 172.21.15.42:0/2880145951 wait complete. 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff4247b3700 1 Processor -- start 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff4247b3700 1 -- start start 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff4247b3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff41c1014a0 0x7ff41c075330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff4247b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c102690 0x7ff41c0739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff4247b3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff41c103030 0x7ff41c073f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff4247b3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c0746a0 con 0x7ff41c102690 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff4247b3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c074810 con 0x7ff41c103030 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff4247b3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41c1a1560 con 0x7ff41c1014a0 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff421d4e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c102690 0x7ff41c0739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff421d4e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c102690 0x7ff41c0739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33520/0 (socket says 172.21.15.42:33520) 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff421d4e700 1 -- 172.21.15.42:0/3689424509 learned_addr learned my addr 172.21.15.42:0/3689424509 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff422d50700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff41c103030 0x7ff41c073f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff42254f700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff41c1014a0 0x7ff41c075330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff421d4e700 1 -- 172.21.15.42:0/3689424509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff41c1014a0 msgr2=0x7ff41c075330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff421d4e700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff41c1014a0 0x7ff41c075330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff421d4e700 1 -- 172.21.15.42:0/3689424509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff41c103030 msgr2=0x7ff41c073f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff421d4e700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff41c103030 0x7ff41c073f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff421d4e700 1 -- 172.21.15.42:0/3689424509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff414009d00 con 0x7ff41c102690 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff421d4e700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c102690 0x7ff41c0739c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c920 tx=0x7ff40c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.768+0000 7ff4137fe700 1 -- 172.21.15.42:0/3689424509 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c007aa0 con 0x7ff41c102690 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.769+0000 7ff4137fe700 1 -- 172.21.15.42:0/3689424509 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff40c00ce70 con 0x7ff41c102690 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.769+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41c1a17f0 con 0x7ff41c102690 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.769+0000 7ff4137fe700 1 -- 172.21.15.42:0/3689424509 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40c005440 con 0x7ff41c102690 2023-12-12T09:56:44.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.769+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41c1a1d00 con 0x7ff41c102690 2023-12-12T09:56:44.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.770+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff41c1088d0 con 0x7ff41c102690 2023-12-12T09:56:44.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.771+0000 7ff4137fe700 1 -- 172.21.15.42:0/3689424509 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7ff40c0055a0 con 0x7ff41c102690 2023-12-12T09:56:44.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.772+0000 7ff4137fe700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4080a1480 0x7ff4080a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:44.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.772+0000 7ff42254f700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4080a1480 0x7ff4080a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:44.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.772+0000 7ff4137fe700 1 -- 172.21.15.42:0/3689424509 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff40c0cce50 con 0x7ff41c102690 2023-12-12T09:56:44.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.772+0000 7ff42254f700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4080a1480 0x7ff4080a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff418005950 tx=0x7ff4180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:44.777 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:44.775+0000 7ff4137fe700 1 -- 172.21.15.42:0/3689424509 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff40c0962c0 con 0x7ff41c102690 2023-12-12T09:56:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:44 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1832868174' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-12T09:56:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:44 smithi173 ceph-mon[103308]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:56:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:44 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1255282620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:44 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/909848740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:44.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:44 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1832868174' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-12T09:56:44.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:44 smithi111 ceph-mon[102565]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:56:44.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:44 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1255282620' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:44.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:44 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/909848740' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:45.025 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.023+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff41c10cf40 con 0x7ff41c102690 2023-12-12T09:56:45.025 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.024+0000 7ff4137fe700 1 -- 172.21.15.42:0/3689424509 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+11270 (secure 0 0 0) 0x7ff40c01d020 con 0x7ff41c102690 2023-12-12T09:56:45.026 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:45.026 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14616,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14628,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24425,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24430,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24452,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24475,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24482,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24508,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T09:56:42.948706+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":24524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-12T09:56:45.028 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4080a1480 msgr2=0x7ff4080a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.028 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4080a1480 0x7ff4080a3940 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff418005950 tx=0x7ff4180058e0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.028 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c102690 msgr2=0x7ff41c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.028 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c102690 0x7ff41c0739c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c920 tx=0x7ff40c00cc30 comp rx=0 tx=0).stop 2023-12-12T09:56:45.028 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 shutdown_connections 2023-12-12T09:56:45.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4080a1480 0x7ff4080a3940 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff41c103030 0x7ff41c073f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff41c1014a0 0x7ff41c075330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 --2- 172.21.15.42:0/3689424509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff41c102690 0x7ff41c0739c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 >> 172.21.15.42:0/3689424509 conn(0x7ff41c0fca30 msgr2=0x7ff41c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:45.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 shutdown_connections 2023-12-12T09:56:45.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.026+0000 7ff4247b3700 1 -- 172.21.15.42:0/3689424509 wait complete. 2023-12-12T09:56:45.029 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 17 2023-12-12T09:56:45.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:56:45.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 -- 172.21.15.42:0/188102297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84101f10 msgr2=0x7f0e841087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 --2- 172.21.15.42:0/188102297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84101f10 0x7f0e841087c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00b0d0 tx=0x7f0e7c00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 -- 172.21.15.42:0/188102297 shutdown_connections 2023-12-12T09:56:45.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 --2- 172.21.15.42:0/188102297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84101f10 0x7f0e841087c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 --2- 172.21.15.42:0/188102297 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e84100380 0x7f0e841007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 --2- 172.21.15.42:0/188102297 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e84101570 0x7f0e841019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 -- 172.21.15.42:0/188102297 >> 172.21.15.42:0/188102297 conn(0x7f0e840fb8f0 msgr2=0x7f0e840fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 -- 172.21.15.42:0/188102297 shutdown_connections 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 -- 172.21.15.42:0/188102297 wait complete. 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.164+0000 7f0e8b986700 1 Processor -- start 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e8b986700 1 -- start start 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e8b986700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84100380 0x7f0e84069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e8b986700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e84101570 0x7f0e8419c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e8b986700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e84101f10 0x7f0e8419e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e8b986700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8419f050 con 0x7f0e84100380 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e8b986700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8419f1c0 con 0x7f0e84101f10 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e8b986700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8419f380 con 0x7f0e84101570 2023-12-12T09:56:45.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89722700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84100380 0x7f0e84069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89722700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84100380 0x7f0e84069150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33532/0 (socket says 172.21.15.42:33532) 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89722700 1 -- 172.21.15.42:0/1948716092 learned_addr learned my addr 172.21.15.42:0/1948716092 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89f23700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e84101f10 0x7f0e8419e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e88f21700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e84101570 0x7f0e8419c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89722700 1 -- 172.21.15.42:0/1948716092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e84101570 msgr2=0x7f0e8419c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89722700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e84101570 0x7f0e8419c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89722700 1 -- 172.21.15.42:0/1948716092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e84101f10 msgr2=0x7f0e8419e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89722700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e84101f10 0x7f0e8419e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89722700 1 -- 172.21.15.42:0/1948716092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c009d00 con 0x7f0e84100380 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e89722700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84100380 0x7f0e84069150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400d980 tx=0x7f0e7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.165+0000 7f0e7a7fc700 1 -- 172.21.15.42:0/1948716092 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400b950 con 0x7f0e84100380 2023-12-12T09:56:45.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.166+0000 7f0e7a7fc700 1 -- 172.21.15.42:0/1948716092 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e74007620 con 0x7f0e84100380 2023-12-12T09:56:45.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.166+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8419f600 con 0x7f0e84100380 2023-12-12T09:56:45.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.166+0000 7f0e7a7fc700 1 -- 172.21.15.42:0/1948716092 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7400f3c0 con 0x7f0e84100380 2023-12-12T09:56:45.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.166+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8419fb10 con 0x7f0e84100380 2023-12-12T09:56:45.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.166+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e84107730 con 0x7f0e84100380 2023-12-12T09:56:45.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.168+0000 7f0e7a7fc700 1 -- 172.21.15.42:0/1948716092 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f0e74010640 con 0x7f0e84100380 2023-12-12T09:56:45.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.168+0000 7f0e7a7fc700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0e700a1480 0x7f0e700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.168+0000 7f0e88f21700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0e700a1480 0x7f0e700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.168+0000 7f0e7a7fc700 1 -- 172.21.15.42:0/1948716092 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0e740cd670 con 0x7f0e84100380 2023-12-12T09:56:45.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.169+0000 7f0e88f21700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0e700a1480 0x7f0e700a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0e840696f0 tx=0x7f0e80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:45.173 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.172+0000 7f0e7a7fc700 1 -- 172.21.15.42:0/1948716092 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f0e74096ae0 con 0x7f0e84100380 2023-12-12T09:56:45.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.376+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e8410bbe0 con 0x7f0e84100380 2023-12-12T09:56:45.380 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.378+0000 7f0e7a7fc700 1 -- 172.21.15.42:0/1948716092 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7f0e7409a100 con 0x7f0e84100380 2023-12-12T09:56:45.381 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:45.381 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:56:45.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0e700a1480 msgr2=0x7f0e700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0e700a1480 0x7f0e700a3940 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0e840696f0 tx=0x7f0e80009260 comp rx=0 tx=0).stop 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84100380 msgr2=0x7f0e84069150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84100380 0x7f0e84069150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400d980 tx=0x7f0e7400dc90 comp rx=0 tx=0).stop 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 shutdown_connections 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f0e700a1480 0x7f0e700a3940 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e84101f10 0x7f0e8419e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0e84100380 0x7f0e84069150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 --2- 172.21.15.42:0/1948716092 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e84101570 0x7f0e8419c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 >> 172.21.15.42:0/1948716092 conn(0x7f0e840fb8f0 msgr2=0x7f0e84104870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 shutdown_connections 2023-12-12T09:56:45.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.380+0000 7f0e8b986700 1 -- 172.21.15.42:0/1948716092 wait complete. 2023-12-12T09:56:45.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-12T09:56:37.902338+0000', 'modified': '2023-12-12T09:56:42.948706+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6839/2430709127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2430709127}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2430709127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-12-12T09:56:45.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-12T09:56:45.395 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-12T09:56:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.522+0000 7fd7780cc700 1 -- 172.21.15.42:0/410288213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770102000 msgr2=0x7fd7701043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.522+0000 7fd7780cc700 1 --2- 172.21.15.42:0/410288213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770102000 0x7fd7701043f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009990 tx=0x7fd76c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.522+0000 7fd7780cc700 1 -- 172.21.15.42:0/410288213 shutdown_connections 2023-12-12T09:56:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.522+0000 7fd7780cc700 1 --2- 172.21.15.42:0/410288213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd770107260 0x7fd7701096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.522+0000 7fd7780cc700 1 --2- 172.21.15.42:0/410288213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770102000 0x7fd7701043f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.522+0000 7fd7780cc700 1 --2- 172.21.15.42:0/410288213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd770104930 0x7fd770106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.522+0000 7fd7780cc700 1 -- 172.21.15.42:0/410288213 >> 172.21.15.42:0/410288213 conn(0x7fd7700fb9f0 msgr2=0x7fd7700fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.522+0000 7fd7780cc700 1 -- 172.21.15.42:0/410288213 shutdown_connections 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.522+0000 7fd7780cc700 1 -- 172.21.15.42:0/410288213 wait complete. 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd7780cc700 1 Processor -- start 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd7780cc700 1 -- start start 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd7780cc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd770102000 0x7fd770101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd7780cc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd770104930 0x7fd7700fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd7780cc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770107260 0x7fd7701004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd7780cc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd770100c90 con 0x7fd770107260 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd7780cc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd770100e00 con 0x7fd770104930 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd7780cc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7701a1500 con 0x7fd770102000 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd776669700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770107260 0x7fd7701004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd776669700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770107260 0x7fd7701004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33548/0 (socket says 172.21.15.42:33548) 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd776669700 1 -- 172.21.15.42:0/1714153475 learned_addr learned my addr 172.21.15.42:0/1714153475 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:45.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd775667700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd770104930 0x7fd7700fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd775e68700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd770102000 0x7fd770101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd776669700 1 -- 172.21.15.42:0/1714153475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd770102000 msgr2=0x7fd770101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd776669700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd770102000 0x7fd770101920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd776669700 1 -- 172.21.15.42:0/1714153475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd770104930 msgr2=0x7fd7700fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd776669700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd770104930 0x7fd7700fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd776669700 1 -- 172.21.15.42:0/1714153475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c0095f0 con 0x7fd770107260 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.523+0000 7fd776669700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770107260 0x7fd7701004f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd768007ee0 tx=0x7fd76800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.524+0000 7fd762ffd700 1 -- 172.21.15.42:0/1714153475 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768017070 con 0x7fd770107260 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.524+0000 7fd762ffd700 1 -- 172.21.15.42:0/1714153475 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd76800f040 con 0x7fd770107260 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.524+0000 7fd7780cc700 1 -- 172.21.15.42:0/1714153475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7701a1790 con 0x7fd770107260 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.524+0000 7fd762ffd700 1 -- 172.21.15.42:0/1714153475 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768005740 con 0x7fd770107260 2023-12-12T09:56:45.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.524+0000 7fd7780cc700 1 -- 172.21.15.42:0/1714153475 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7701a1ca0 con 0x7fd770107260 2023-12-12T09:56:45.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.526+0000 7fd762ffd700 1 -- 172.21.15.42:0/1714153475 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd7680058a0 con 0x7fd770107260 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.526+0000 7fd762ffd700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd75c0a1500 0x7fd75c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.527+0000 7fd775e68700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd75c0a1500 0x7fd75c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.527+0000 7fd762ffd700 1 -- 172.21.15.42:0/1714153475 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd7680cc730 con 0x7fd770107260 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.527+0000 7fd775e68700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd75c0a1500 0x7fd75c0a39c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00b5c0 tx=0x7fd76c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.527+0000 7fd7842c5b80 1 Processor -- start 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.527+0000 7fd7842c5b80 1 -- start start 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780380 0x5590a17807a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780ce0 0x5590a17815f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1781bd0 0x5590a1782050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a16dbb00 con 0x5590a1781bd0 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a16d9420 con 0x5590a1780ce0 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a16d4d10 con 0x5590a1780380 2023-12-12T09:56:45.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd776669700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1781bd0 0x5590a1782050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd776669700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1781bd0 0x5590a1782050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33556/0 (socket says 172.21.15.42:33556) 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd776669700 1 -- 172.21.15.42:0/1974015199 learned_addr learned my addr 172.21.15.42:0/1974015199 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd775667700 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780ce0 0x5590a17815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd775e68700 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780380 0x5590a17807a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd776669700 1 -- 172.21.15.42:0/1974015199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780380 msgr2=0x5590a17807a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd776669700 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780380 0x5590a17807a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd776669700 1 -- 172.21.15.42:0/1974015199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780ce0 msgr2=0x5590a17815f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd776669700 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780ce0 0x5590a17815f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd776669700 1 -- 172.21.15.42:0/1974015199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a15515e0 con 0x5590a1781bd0 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd775e68700 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780380 0x5590a17807a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd775667700 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780ce0 0x5590a17815f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd776669700 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1781bd0 0x5590a1782050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd768095d00 tx=0x7fd76800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0a7505caea60b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd75bfff700 1 -- 172.21.15.42:0/1974015199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7680a1b90 con 0x5590a1781bd0 2023-12-12T09:56:45.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd75bfff700 1 -- 172.21.15.42:0/1974015199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd768097ba0 con 0x5590a1781bd0 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.528+0000 7fd75bfff700 1 -- 172.21.15.42:0/1974015199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd768099c80 con 0x5590a1781bd0 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.529+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1974015199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1781bd0 msgr2=0x5590a1782050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.529+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1781bd0 0x5590a1782050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd768095d00 tx=0x7fd76800dc00 comp rx=0 tx=0).stop 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.529+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1974015199 shutdown_connections 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.529+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1781bd0 0x5590a1782050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.529+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780380 0x5590a17807a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.529+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1974015199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780ce0 0x5590a17815f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.529+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1974015199 >> 172.21.15.42:0/1974015199 conn(0x5590a16af790 msgr2=0x5590a177cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.529+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1974015199 shutdown_connections 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.529+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1974015199 wait complete. 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd7842c5b80 1 -- start start 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd7842c5b80 1 Processor -- start 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 0x5590a178a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 0x5590a1791760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a168e510 con 0x5590a1780ce0 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a168e1b0 con 0x5590a1780380 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a15538d0 con 0x5590a1781bd0 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd775667700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd7842c5b80 10 client.0 _ll_register_callbacks cb 0x5590a15b9a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd775667700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33560/0 (socket says 172.21.15.42:33560) 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.530+0000 7fd775667700 1 -- 172.21.15.42:0/1937533657 learned_addr learned my addr 172.21.15.42:0/1937533657 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd775e68700 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 0x5590a178a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd776669700 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 0x5590a1791760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd775667700 1 -- 172.21.15.42:0/1937533657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 msgr2=0x5590a1791760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd775667700 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 0x5590a1791760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd775667700 1 -- 172.21.15.42:0/1937533657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 msgr2=0x5590a178a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:56:45.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd775667700 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 0x5590a178a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd775667700 1 -- 172.21.15.42:0/1937533657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7701a1ca0 con 0x5590a1780ce0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd776669700 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 0x5590a1791760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd775667700 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178a720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd76400bfd0 tx=0x7fd764011740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d792677e3b6e359 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd7597fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd7597fa700 1 -- 172.21.15.42:0/1937533657 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764004230 con 0x5590a1780ce0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1937533657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590a1773af0 con 0x5590a1780ce0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd7597fa700 1 -- 172.21.15.42:0/1937533657 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd764011e70 con 0x5590a1780ce0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd7597fa700 1 -- 172.21.15.42:0/1937533657 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd764004390 con 0x5590a1780ce0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd7842c5b80 10 client.14748 fetch_fsmap learned FSMap version 17 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1937533657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5590a14fdd90 con 0x5590a1780ce0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.531+0000 7fd7597fa700 1 -- 172.21.15.42:0/1937533657 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590a1773af0 con 0x5590a1780ce0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd7597fa700 1 -- 172.21.15.42:0/1937533657 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd76401b6f0 con 0x5590a1780ce0 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd7842c5b80 10 client.14748 fetch_fsmap finished waiting for FSMap version 17 2023-12-12T09:56:45.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd7842c5b80 10 client.14748 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:56:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd7842c5b80 20 client.14748 populate_metadata read hostname 'smithi042' 2023-12-12T09:56:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5590a1795ee0 0x5590a17949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd7842c5b80 4 client.14748 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:56:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1937533657 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590a174fef0 con 0x5590a1795ee0 2023-12-12T09:56:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd775e68700 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5590a1795ee0 0x5590a17949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd7597fa700 10 client.14748 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:56:45.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.532+0000 7fd775e68700 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5590a1795ee0 0x5590a17949b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6137f3d13bf21961 server_cookie=ff9976a3ea160fb9 in_seq=0 out_seq=0 2023-12-12T09:56:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7597fa700 1 -- 172.21.15.42:0/1937533657 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590a174fef0 con 0x5590a1795ee0 2023-12-12T09:56:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7597fa700 10 client.14748 handle_command_reply: tid=0 2023-12-12T09:56:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 client.14748 shutdown 2023-12-12T09:56:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1937533657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 msgr2=0x5590a178a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178a720 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd76400bfd0 tx=0x7fd764011740 comp rx=0 tx=0).stop 2023-12-12T09:56:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1937533657 shutdown_connections 2023-12-12T09:56:45.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5590a1795ee0 0x5590a17949b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 0x5590a178a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 0x5590a1791760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1937533657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178a720 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7597fa700 0 client.14748 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:56:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1937533657 >> 172.21.15.42:0/1937533657 conn(0x5590a16af790 msgr2=0x5590a177b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.533+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1937533657 shutdown_connections 2023-12-12T09:56:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.534+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1937533657 wait complete. 2023-12-12T09:56:45.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.534+0000 7fd7842c5b80 20 client.14748 trim_cache size 0 max 16384 2023-12-12T09:56:45.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.551+0000 7fd7842c5b80 1 Processor -- start 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.551+0000 7fd7842c5b80 1 -- start start 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780380 0x5590a177c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780ce0 0x5590a177ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1781bd0 0x5590a1791720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a177afd0 con 0x5590a1780380 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a1781480 con 0x5590a1781bd0 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a16d5fa0 con 0x5590a1780ce0 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775e68700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780380 0x5590a177c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775e68700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780380 0x5590a177c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33566/0 (socket says 172.21.15.42:33566) 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775e68700 1 -- 172.21.15.42:0/1251406346 learned_addr learned my addr 172.21.15.42:0/1251406346 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd776669700 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1781bd0 0x5590a1791720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775667700 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780ce0 0x5590a177ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775e68700 1 -- 172.21.15.42:0/1251406346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780ce0 msgr2=0x5590a177ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775e68700 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780ce0 0x5590a177ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775e68700 1 -- 172.21.15.42:0/1251406346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1781bd0 msgr2=0x5590a1791720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775e68700 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1781bd0 0x5590a1791720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775e68700 1 -- 172.21.15.42:0/1251406346 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590a14e6c10 con 0x5590a1780380 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd776669700 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1781bd0 0x5590a1791720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775667700 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780ce0 0x5590a177ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd775e68700 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780380 0x5590a177c920 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd76c000c00 tx=0x7fd76c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7970a567bf69db9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd760ff9700 1 -- 172.21.15.42:0/1251406346 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0108d0 con 0x5590a1780380 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd760ff9700 1 -- 172.21.15.42:0/1251406346 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd76c02c040 con 0x5590a1780380 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.552+0000 7fd760ff9700 1 -- 172.21.15.42:0/1251406346 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c024410 con 0x5590a1780380 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1251406346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780380 msgr2=0x5590a177c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780380 0x5590a177c920 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd76c000c00 tx=0x7fd76c01f000 comp rx=0 tx=0).stop 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1251406346 shutdown_connections 2023-12-12T09:56:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1781bd0 0x5590a1791720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780380 0x5590a177c920 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/1251406346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1780ce0 0x5590a177ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1251406346 >> 172.21.15.42:0/1251406346 conn(0x5590a16af790 msgr2=0x5590a17896f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1251406346 shutdown_connections 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 -- 172.21.15.42:0/1251406346 wait complete. 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 -- start start 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.553+0000 7fd7842c5b80 1 Processor -- start 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 0x5590a1791350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd7842c5b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 0x5590a178e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a1791b70 con 0x5590a1780ce0 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a178e900 con 0x5590a1780380 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd7842c5b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590a178eac0 con 0x5590a1781bd0 2023-12-12T09:56:45.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd7842c5b80 10 client.0 _ll_register_callbacks cb 0x5590a15b9a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd775667700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd775667700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33574/0 (socket says 172.21.15.42:33574) 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd775667700 1 -- 172.21.15.42:0/4197430071 learned_addr learned my addr 172.21.15.42:0/4197430071 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd775e68700 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 0x5590a1791350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd776669700 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 0x5590a178e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd775667700 1 -- 172.21.15.42:0/4197430071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 msgr2=0x5590a178e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd775667700 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 0x5590a178e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd775667700 1 -- 172.21.15.42:0/4197430071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 msgr2=0x5590a1791350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd775667700 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 0x5590a1791350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.554+0000 7fd775667700 1 -- 172.21.15.42:0/4197430071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77010a2c0 con 0x5590a1780ce0 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd775667700 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178de50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd764012470 tx=0x7fd764011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94a4f5d161b59ba server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7597fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7597fa700 1 -- 172.21.15.42:0/4197430071 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764005240 con 0x5590a1780ce0 2023-12-12T09:56:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7597fa700 1 -- 172.21.15.42:0/4197430071 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd764000e40 con 0x5590a1780ce0 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7842c5b80 1 -- 172.21.15.42:0/4197430071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd768097ba0 con 0x5590a1780ce0 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7597fa700 1 -- 172.21.15.42:0/4197430071 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd76400b3a0 con 0x5590a1780ce0 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7842c5b80 10 client.14754 fetch_fsmap learned FSMap version 17 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7842c5b80 1 -- 172.21.15.42:0/4197430071 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5590a1524200 con 0x5590a1780ce0 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7597fa700 1 -- 172.21.15.42:0/4197430071 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd764036020 con 0x5590a1780ce0 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7597fa700 1 -- 172.21.15.42:0/4197430071 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 17) v1 ==== 7902+0+0 (secure 0 0 0) 0x7fd764029070 con 0x5590a1780ce0 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7842c5b80 10 client.14754 fetch_fsmap finished waiting for FSMap version 17 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7842c5b80 10 client.14754 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7842c5b80 20 client.14754 populate_metadata read hostname 'smithi042' 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5590a17f1180 0x5590a17ed620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7842c5b80 4 client.14754 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.555+0000 7fd7842c5b80 1 -- 172.21.15.42:0/4197430071 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5590a169af80 con 0x5590a17f1180 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd775e68700 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5590a17f1180 0x5590a17ed620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7597fa700 10 client.14754 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:56:45.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd775e68700 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5590a17f1180 0x5590a17ed620 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38aabcf8400e86c2 server_cookie=691bd73a213098d3 in_seq=0 out_seq=0 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7597fa700 1 -- 172.21.15.42:0/4197430071 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5590a169af80 con 0x5590a17f1180 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7597fa700 10 client.14754 handle_command_reply: tid=0 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 client.14754 shutdown 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 -- 172.21.15.42:0/4197430071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 msgr2=0x5590a178de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178de50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd764012470 tx=0x7fd764011b40 comp rx=0 tx=0).stop 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 -- 172.21.15.42:0/4197430071 shutdown_connections 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5590a17f1180 0x5590a17ed620 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5590a1780380 0x5590a1791350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7597fa700 0 client.14754 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5590a1781bd0 0x5590a178e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 --2- 172.21.15.42:0/4197430071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5590a1780ce0 0x5590a178de50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 -- 172.21.15.42:0/4197430071 >> 172.21.15.42:0/4197430071 conn(0x5590a16af790 msgr2=0x5590a1784bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 -- 172.21.15.42:0/4197430071 shutdown_connections 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 1 -- 172.21.15.42:0/4197430071 wait complete. 2023-12-12T09:56:45.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.556+0000 7fd7842c5b80 20 client.14754 trim_cache size 0 max 16384 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24524, 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 7.0963749649999999, 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 17, 2023-12-12T09:56:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 87, 2023-12-12T09:56:45.567 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:56:45.567 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 13.175542987 2023-12-12T09:56:45.567 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 -- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd75c0a1500 msgr2=0x7fd75c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd75c0a1500 0x7fd75c0a39c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd76c00b5c0 tx=0x7fd76c005fb0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 -- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770107260 msgr2=0x7fd7701004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770107260 0x7fd7701004f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd768007ee0 tx=0x7fd76800d3b0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 -- 172.21.15.42:0/1714153475 shutdown_connections 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd75c0a1500 0x7fd75c0a39c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd770107260 0x7fd7701004f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd770102000 0x7fd770101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 --2- 172.21.15.42:0/1714153475 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd770104930 0x7fd7700fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 -- 172.21.15.42:0/1714153475 >> 172.21.15.42:0/1714153475 conn(0x7fd7700fb9f0 msgr2=0x7fd770102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 -- 172.21.15.42:0/1714153475 shutdown_connections 2023-12-12T09:56:45.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:45.557+0000 7fd7780cc700 1 -- 172.21.15.42:0/1714153475 wait complete. 2023-12-12T09:56:45.571 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-12T09:56:45.571 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:56:45.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph fs ls 2023-12-12T09:56:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:45 smithi042 ceph-mon[98894]: pgmap v265: 129 pgs: 2 creating+activating, 11 creating+peering, 103 active+clean, 13 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2023-12-12T09:56:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:45 smithi042 ceph-mon[98894]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-12T09:56:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:45 smithi042 ceph-mon[98894]: Cluster is now healthy 2023-12-12T09:56:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:45 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/840826137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:45.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:45 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3689424509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:45.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:45 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1948716092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:45 smithi173 ceph-mon[103308]: pgmap v265: 129 pgs: 2 creating+activating, 11 creating+peering, 103 active+clean, 13 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2023-12-12T09:56:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:45 smithi173 ceph-mon[103308]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-12T09:56:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:45 smithi173 ceph-mon[103308]: Cluster is now healthy 2023-12-12T09:56:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:45 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/840826137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:45 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3689424509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:45 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1948716092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:45.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:45 smithi111 ceph-mon[102565]: pgmap v265: 129 pgs: 2 creating+activating, 11 creating+peering, 103 active+clean, 13 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s wr, 8 op/s 2023-12-12T09:56:45.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:45 smithi111 ceph-mon[102565]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2023-12-12T09:56:45.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:45 smithi111 ceph-mon[102565]: Cluster is now healthy 2023-12-12T09:56:45.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:45 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/840826137' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:45.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:45 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3689424509' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:56:45.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:45 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1948716092' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:46.168 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:56:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.372+0000 7f53e58cb700 1 -- 172.21.15.42:0/4206423807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 msgr2=0x7f53e0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.372+0000 7f53e58cb700 1 --2- 172.21.15.42:0/4206423807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 0x7f53e0106940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f53d00099c0 tx=0x7f53d0009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.372+0000 7f53e58cb700 1 -- 172.21.15.42:0/4206423807 shutdown_connections 2023-12-12T09:56:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.372+0000 7f53e58cb700 1 --2- 172.21.15.42:0/4206423807 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e0100ef0 0x7f53e010d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.372+0000 7f53e58cb700 1 --2- 172.21.15.42:0/4206423807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 0x7f53e0106940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.375 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.372+0000 7f53e58cb700 1 --2- 172.21.15.42:0/4206423807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e0100530 0x7f53e01009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.376 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.372+0000 7f53e58cb700 1 -- 172.21.15.42:0/4206423807 >> 172.21.15.42:0/4206423807 conn(0x7f53e0076570 msgr2=0x7f53e0076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:47.376 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.372+0000 7f53e58cb700 1 -- 172.21.15.42:0/4206423807 shutdown_connections 2023-12-12T09:56:47.376 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.372+0000 7f53e58cb700 1 -- 172.21.15.42:0/4206423807 wait complete. 2023-12-12T09:56:47.376 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53e58cb700 1 Processor -- start 2023-12-12T09:56:47.376 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53e58cb700 1 -- start start 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53e58cb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e0100530 0x7f53e0198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53e58cb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e0100ef0 0x7f53e0198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53e58cb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 0x7f53e01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53e58cb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01a3b40 con 0x7f53e0106560 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53e58cb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01a3cb0 con 0x7f53e0100530 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53e58cb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e01a3e70 con 0x7f53e0100ef0 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53df7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 0x7f53e01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.373+0000 7f53df7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 0x7f53e01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33590/0 (socket says 172.21.15.42:33590) 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53df7fe700 1 -- 172.21.15.42:0/3536825756 learned_addr learned my addr 172.21.15.42:0/3536825756 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:47.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53deffd700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e0100530 0x7f53e0198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53de7fc700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e0100ef0 0x7f53e0198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53df7fe700 1 -- 172.21.15.42:0/3536825756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e0100ef0 msgr2=0x7f53e0198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53df7fe700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e0100ef0 0x7f53e0198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53df7fe700 1 -- 172.21.15.42:0/3536825756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e0100530 msgr2=0x7f53e0198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53df7fe700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e0100530 0x7f53e0198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53df7fe700 1 -- 172.21.15.42:0/3536825756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d00095f0 con 0x7f53e0106560 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53df7fe700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 0x7f53e01992f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53d8007ee0 tx=0x7f53d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53e48c9700 1 -- 172.21.15.42:0/3536825756 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8017070 con 0x7f53e0106560 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53e48c9700 1 -- 172.21.15.42:0/3536825756 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f53d800f040 con 0x7f53e0106560 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e01a4150 con 0x7f53e0106560 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53e48c9700 1 -- 172.21.15.42:0/3536825756 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8005740 con 0x7f53e0106560 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.374+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e01a4650 con 0x7f53e0106560 2023-12-12T09:56:47.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.375+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e01089f0 con 0x7f53e0106560 2023-12-12T09:56:47.380 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.376+0000 7f53e48c9700 1 -- 172.21.15.42:0/3536825756 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f53d800dc60 con 0x7f53e0106560 2023-12-12T09:56:47.380 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.377+0000 7f53e48c9700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f53cc0a1500 0x7f53cc0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:47.380 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.377+0000 7f53e48c9700 1 -- 172.21.15.42:0/3536825756 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f53d80cc690 con 0x7f53e0106560 2023-12-12T09:56:47.380 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.377+0000 7f53deffd700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f53cc0a1500 0x7f53cc0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:47.380 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.377+0000 7f53deffd700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f53cc0a1500 0x7f53cc0a39c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009990 tx=0x7f53d0005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:47.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.381+0000 7f53e48c9700 1 -- 172.21.15.42:0/3536825756 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f53d8095b00 con 0x7f53e0106560 2023-12-12T09:56:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.623+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f53e004f290 con 0x7f53e0106560 2023-12-12T09:56:47.626 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.623+0000 7f53e48c9700 1 -- 172.21.15.42:0/3536825756 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f53d8099120 con 0x7f53e0106560 2023-12-12T09:56:47.626 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-12T09:56:47.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f53cc0a1500 msgr2=0x7f53cc0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f53cc0a1500 0x7f53cc0a39c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f53d0009990 tx=0x7f53d0005a90 comp rx=0 tx=0).stop 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 msgr2=0x7f53e01992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 0x7f53e01992f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f53d8007ee0 tx=0x7f53d800d3b0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 shutdown_connections 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f53cc0a1500 0x7f53cc0a39c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f53e0100530 0x7f53e0198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f53e0106560 0x7f53e01992f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 --2- 172.21.15.42:0/3536825756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53e0100ef0 0x7f53e0198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.625+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 >> 172.21.15.42:0/3536825756 conn(0x7f53e0076570 msgr2=0x7f53e00ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.626+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 shutdown_connections 2023-12-12T09:56:47.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:47.626+0000 7f53e58cb700 1 -- 172.21.15.42:0/3536825756 wait complete. 2023-12-12T09:56:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:47 smithi173 ceph-mon[103308]: pgmap v266: 129 pgs: 2 creating+activating, 11 creating+peering, 112 active+clean, 4 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2023-12-12T09:56:47.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:47 smithi042 ceph-mon[98894]: pgmap v266: 129 pgs: 2 creating+activating, 11 creating+peering, 112 active+clean, 4 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2023-12-12T09:56:47.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:47 smithi111 ceph-mon[102565]: pgmap v266: 129 pgs: 2 creating+activating, 11 creating+peering, 112 active+clean, 4 unknown; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2023-12-12T09:56:48.030 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-12T09:56:48.030 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-12T09:56:48.030 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2023-12-12T09:56:48.030 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-12-12T09:56:48.030 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:56:48.030 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-12T09:56:48.030 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-12T09:56:48.031 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-12T09:56:48.031 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-12-12T09:56:48.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.031 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2023-12-12T09:56:48.049 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:56:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:56:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:56:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:56:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2023-12-12T09:56:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:56:48.050 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:56:48.050 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2023-12-12T09:56:48.050 INFO:teuthology.orchestra.run.smithi042.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-12T09:56:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:56:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-12T09:56:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41828sec preferred_lft 41828sec 2023-12-12T09:56:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2023-12-12T09:56:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:56:48.051 INFO:teuthology.orchestra.run.smithi042.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:56:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2023-12-12T09:56:48.051 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:56:48.051 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:56:48.052 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-12T09:56:48.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.052 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:56:48.052 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:56:48.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2023-12-12T09:56:48.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2023-12-12T09:56:48.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2023-12-12T09:56:48.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-12T09:56:48.052 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:56:48.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.231 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-12T09:56:48.264 INFO:teuthology.orchestra.run.smithi042.stdout:1 2023-12-12T09:56:48.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.265 DEBUG:teuthology.orchestra.run.smithi042:> route 2023-12-12T09:56:48.352 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2023-12-12T09:56:48.352 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-12T09:56:48.352 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-12T09:56:48.352 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-12T09:56:48.352 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-12T09:56:48.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.353 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:56:48.353 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:56:48.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-12T09:56:48.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-12T09:56:48.353 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-12T09:56:48.353 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:56:48.537 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:48 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3536825756' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-12T09:56:48.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.605 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2023-12-12T09:56:48.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.623 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2023-12-12T09:56:48.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.680 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:56:48.680 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:56:48.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:56:48.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-12-12T09:56:48.681 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:56:48.778 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-12-12T09:56:48.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.778 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:56:48.778 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:56:48.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-12-12T09:56:48.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-12T09:56:48.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-12-12T09:56:48.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-12-12T09:56:48.779 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-12-12T09:56:48.779 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:56:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:48 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3536825756' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-12T09:56:48.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:48.968 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:56:48.968 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:56:48.968 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2023-12-12T09:56:48.968 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2023-12-12T09:56:48.968 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:56:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:48 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3536825756' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-12T09:56:49.067 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-12-12T09:56:49.067 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:56:49.067 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:56:49.085 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-12-12T09:56:49.085 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:56:49.085 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:56:49.143 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2023-12-12T09:56:49.212 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2023-12-12T09:56:49.270 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2023-12-12T09:56:49.270 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2023-12-12T09:56:49.270 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2023-12-12T09:56:49.270 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2023-12-12T09:56:49.270 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2023-12-12T09:56:49.270 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/systemd 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/rdma 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/cpuset 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/freezer 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/memory 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/pids 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/hugetlb 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-12-12T09:56:49.271 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/devices 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/perf_event 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/blkio 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2023-12-12T09:56:49.272 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/6f8e8cc00fbc3287931424677c389b899590bcf5658760c5ebbac9fa5a6ba10e/merged 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3fa2c115a1fbdcde5db3a6b2e3f1a23195b03c5d442b1d58902d9505d0f29042/merged 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/236d7e635fd15bc6af4e50d772bbc6b8585460643e4fe16e5c5654be562db345/merged 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/985b0f4d1c117ede694f6a6544a4c8aba3901acb0e3b4bd895b0785220b22ac9/merged 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/7f025a362b3a716b3734b5781d0bd46bda6b357c17adfb167aba9a164730c884/merged 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/e88936f7db214cd427d78b4260113ab9cd9d1ba487e054933ab8d80d88eb50dd/merged 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/5612dcd2ad76da1ba772d050d66be59e0723670be9390106fded9a8278ef8ef3/merged 2023-12-12T09:56:49.273 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4ee770653b98b781911b7d6a4ee3bb683b80805b6dea4b66fbf954359fb4b88b/merged 2023-12-12T09:56:49.274 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/61519035d9f6c371d984841d6fca86574746e0d6e88447d19dd29bf96ee8c790/merged 2023-12-12T09:56:49.274 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/6e141e6a667844791a62c18d62582682b91c261ae2f82143c678e717b34205c8/merged 2023-12-12T09:56:49.274 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2023-12-12T09:56:49.274 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:56:49.274 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:56:49.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:49.274 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2023-12-12T09:56:49.328 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-12T09:56:49.328 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-12-12T09:56:49.371 DEBUG:teuthology.orchestra.run.smithi042:> sudo modprobe fuse 2023-12-12T09:56:49.399 DEBUG:teuthology.orchestra.run.smithi042:> cat /proc/self/mounts | awk '{print $2}' 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/sys 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/proc 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/dev 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/security 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/shm 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/pts 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/run 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/systemd 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/pstore 2023-12-12T09:56:49.457 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/bpf 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/rdma 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/cpuset 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/freezer 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/memory 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/pids 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/hugetlb 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/devices 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/perf_event 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/cgroup/blkio 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/tracing 2023-12-12T09:56:49.458 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/config 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/ 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/selinux 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/kernel/debug 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/hugepages 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/dev/mqueue 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/sys/fs/fuse/connections 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/nfs/rpc_pipefs 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/1000 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/ceph 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/6f8e8cc00fbc3287931424677c389b899590bcf5658760c5ebbac9fa5a6ba10e/merged 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/3fa2c115a1fbdcde5db3a6b2e3f1a23195b03c5d442b1d58902d9505d0f29042/merged 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/run/user/0 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/proc/sys/fs/binfmt_misc 2023-12-12T09:56:49.459 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/236d7e635fd15bc6af4e50d772bbc6b8585460643e4fe16e5c5654be562db345/merged 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/985b0f4d1c117ede694f6a6544a4c8aba3901acb0e3b4bd895b0785220b22ac9/merged 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/7f025a362b3a716b3734b5781d0bd46bda6b357c17adfb167aba9a164730c884/merged 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/e88936f7db214cd427d78b4260113ab9cd9d1ba487e054933ab8d80d88eb50dd/merged 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/5612dcd2ad76da1ba772d050d66be59e0723670be9390106fded9a8278ef8ef3/merged 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/4ee770653b98b781911b7d6a4ee3bb683b80805b6dea4b66fbf954359fb4b88b/merged 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/61519035d9f6c371d984841d6fca86574746e0d6e88447d19dd29bf96ee8c790/merged 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/var/lib/containers/storage/overlay/6e141e6a667844791a62c18d62582682b91c261ae2f82143c678e717b34205c8/merged 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run.smithi042.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:56:49.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:49.460 DEBUG:teuthology.orchestra.run.smithi042:> ls /sys/fs/fuse/connections 2023-12-12T09:56:49.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi042.stderr:2023-12-12T09:56:49.478+0000 7fd0cd8875c0 -1 init, newargv = 0x5626409b9800 newargc=15 2023-12-12T09:56:49.480 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi042.stderr:ceph-fuse[153974]: starting ceph client 2023-12-12T09:56:49.487 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi042.stderr:ceph-fuse[153974]: starting fuse 2023-12-12T09:56:49.498 INFO:teuthology.orchestra.run.smithi042.stdout:124 2023-12-12T09:56:49.498 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> sudo stdin-killer -- python3 -c ' 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> import glob 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> import re 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> import os 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> import subprocess 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> def _find_admin_socket(client_name): 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> files = glob.glob(asok_path) 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> # Given a non-glob path, it better be there 2023-12-12T09:56:49.499 DEBUG:teuthology.orchestra.run.smithi042:> if "*" not in asok_path: 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> assert(len(files) == 1) 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> return files[0] 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> for f in files: 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> contents = proc_f.read() 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> if mountpoint in contents: 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> return f 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> print(_find_admin_socket("client.admin")) 2023-12-12T09:56:49.500 DEBUG:teuthology.orchestra.run.smithi042:> ' 2023-12-12T09:56:49.626 INFO:teuthology.orchestra.run.smithi042.stdout:/var/run/ceph/ceph-client.admin.153974.asok 2023-12-12T09:56:49.628 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-12T09:56:49.635 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.153974.asok 2023-12-12T09:56:49.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:49.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.153974.asok status 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "metadata": { 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_sha1": "f1fbf3c583d03e19f5e58eb618e78d8e10d075f9", 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-96-gf1fbf3c5 (f1fbf3c583d03e19f5e58eb618e78d8e10d075f9) reef (dev)", 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "entity_id": "admin", 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": "smithi042", 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "pid": "153974", 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: "root": "/" 2023-12-12T09:56:49.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_count": 0, 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "dentry_pinned_count": 0, 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 14766, 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "inst": { 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": { 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "client", 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "num": 14766 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2769170940 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": { 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2023-12-12T09:56:49.799 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "192.168.0.1:0", 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2769170940 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "inst_str": "client.14766 192.168.0.1:0/2769170940", 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "addr_str": "192.168.0.1:0/2769170940", 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "inode_count": 1, 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_epoch": 17, 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch": 87, 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_epoch_barrier": 0, 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "blocklisted": false, 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs" 2023-12-12T09:56:49.800 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:56:49.807 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:49 smithi042 ceph-mon[98894]: pgmap v267: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2023-12-12T09:56:49.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:49.808 DEBUG:teuthology.orchestra.run.smithi042:> stat --file-system '--printf=%T 2023-12-12T09:56:49.808 DEBUG:teuthology.orchestra.run.smithi042:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:56:49.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:49 smithi173 ceph-mon[103308]: pgmap v267: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2023-12-12T09:56:49.825 INFO:teuthology.orchestra.run.smithi042.stdout:fuseblk 2023-12-12T09:56:49.825 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:56:49.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:49.826 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:56:49.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:49.895 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-12-12T09:56:49.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:49 smithi111 ceph-mon[102565]: pgmap v267: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2023-12-12T09:56:50.013 INFO:teuthology.orchestra.run.smithi042.stdout:# file: . 2023-12-12T09:56:50.013 INFO:teuthology.orchestra.run.smithi042.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-12-12T09:56:50.013 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:50.014 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:56:50.014 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:56:50.032 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:56:50.199 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.195+0000 7f9a97019700 1 -- 172.21.15.42:0/3723554452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a901048d0 msgr2=0x7f9a90106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.195+0000 7f9a97019700 1 --2- 172.21.15.42:0/3723554452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a901048d0 0x7f9a90106cc0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009990 tx=0x7f9a7c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.195+0000 7f9a97019700 1 -- 172.21.15.42:0/3723554452 shutdown_connections 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.195+0000 7f9a97019700 1 --2- 172.21.15.42:0/3723554452 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a90101fa0 0x7f9a90104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.195+0000 7f9a97019700 1 --2- 172.21.15.42:0/3723554452 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a90107200 0x7f9a901095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.195+0000 7f9a97019700 1 --2- 172.21.15.42:0/3723554452 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a901048d0 0x7f9a90106cc0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.195+0000 7f9a97019700 1 -- 172.21.15.42:0/3723554452 >> 172.21.15.42:0/3723554452 conn(0x7f9a900fb8f0 msgr2=0x7f9a900fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.195+0000 7f9a97019700 1 -- 172.21.15.42:0/3723554452 shutdown_connections 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.195+0000 7f9a97019700 1 -- 172.21.15.42:0/3723554452 wait complete. 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a97019700 1 Processor -- start 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a97019700 1 -- start start 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a97019700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a90101fa0 0x7f9a901973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a97019700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a901048d0 0x7f9a90197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a97019700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a90107200 0x7f9a9019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a97019700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9019eef0 con 0x7f9a90107200 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a97019700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9019f030 con 0x7f9a901048d0 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a97019700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9019f1c0 con 0x7f9a90101fa0 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a955b6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a90107200 0x7f9a9019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a955b6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a90107200 0x7f9a9019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33614/0 (socket says 172.21.15.42:33614) 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a955b6700 1 -- 172.21.15.42:0/2415861407 learned_addr learned my addr 172.21.15.42:0/2415861407 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a94db5700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a90101fa0 0x7f9a901973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a8ffff700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a901048d0 0x7f9a90197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a955b6700 1 -- 172.21.15.42:0/2415861407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a90101fa0 msgr2=0x7f9a901973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a955b6700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a90101fa0 0x7f9a901973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.196+0000 7f9a955b6700 1 -- 172.21.15.42:0/2415861407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a901048d0 msgr2=0x7f9a90197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a955b6700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a901048d0 0x7f9a90197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a955b6700 1 -- 172.21.15.42:0/2415861407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c0095f0 con 0x7f9a90107200 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a8ffff700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a901048d0 0x7f9a90197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:56:50.201 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a94db5700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a90101fa0 0x7f9a901973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:50.202 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a955b6700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a90107200 0x7f9a9019e920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a88007ee0 tx=0x7f9a8800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:50.202 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a8dffb700 1 -- 172.21.15.42:0/2415861407 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88017070 con 0x7f9a90107200 2023-12-12T09:56:50.202 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a8dffb700 1 -- 172.21.15.42:0/2415861407 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a8800f040 con 0x7f9a90107200 2023-12-12T09:56:50.202 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9019f4a0 con 0x7f9a90107200 2023-12-12T09:56:50.202 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a8dffb700 1 -- 172.21.15.42:0/2415861407 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88005740 con 0x7f9a90107200 2023-12-12T09:56:50.202 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.197+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9019f9b0 con 0x7f9a90107200 2023-12-12T09:56:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.198+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a900fd470 con 0x7f9a90107200 2023-12-12T09:56:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.199+0000 7f9a8dffb700 1 -- 172.21.15.42:0/2415861407 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f9a880058a0 con 0x7f9a90107200 2023-12-12T09:56:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.200+0000 7f9a8dffb700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9a800a15d0 0x7f9a800a3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.200+0000 7f9a94db5700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9a800a15d0 0x7f9a800a3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.200+0000 7f9a8dffb700 1 -- 172.21.15.42:0/2415861407 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f9a880cc6a0 con 0x7f9a90107200 2023-12-12T09:56:50.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.200+0000 7f9a94db5700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9a800a15d0 0x7f9a800a3a90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400bcf0 tx=0x7f9a8400b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:50.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.204+0000 7f9a8dffb700 1 -- 172.21.15.42:0/2415861407 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9a88095b10 con 0x7f9a90107200 2023-12-12T09:56:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.407+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a90068230 con 0x7f9a90107200 2023-12-12T09:56:50.411 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.409+0000 7f9a8dffb700 1 -- 172.21.15.42:0/2415861407 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7f9a88099130 con 0x7f9a90107200 2023-12-12T09:56:50.411 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:50.412 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:56:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9a800a15d0 msgr2=0x7f9a800a3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9a800a15d0 0x7f9a800a3a90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400bcf0 tx=0x7f9a8400b2c0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a90107200 msgr2=0x7f9a9019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a90107200 0x7f9a9019e920 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9a88007ee0 tx=0x7f9a8800d3b0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 shutdown_connections 2023-12-12T09:56:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9a800a15d0 0x7f9a800a3a90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a90107200 0x7f9a9019e920 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a90101fa0 0x7f9a901973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 --2- 172.21.15.42:0/2415861407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a901048d0 0x7f9a90197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 >> 172.21.15.42:0/2415861407 conn(0x7f9a900fb8f0 msgr2=0x7f9a90102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:50.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 shutdown_connections 2023-12-12T09:56:50.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.411+0000 7f9a97019700 1 -- 172.21.15.42:0/2415861407 wait complete. 2023-12-12T09:56:50.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:56:35.884237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:56:37.283813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:56:40.918865+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:56:50.426 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:56:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 -- 172.21.15.42:0/2074875051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 msgr2=0x7f42d01008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 --2- 172.21.15.42:0/2074875051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 0x7f42d01008d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f42c00099c0 tx=0x7f42c0009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 -- 172.21.15.42:0/2074875051 shutdown_connections 2023-12-12T09:56:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 --2- 172.21.15.42:0/2074875051 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d0102040 0x7f42d01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 --2- 172.21.15.42:0/2074875051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 0x7f42d01008d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 --2- 172.21.15.42:0/2074875051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42d01016a0 0x7f42d0101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 -- 172.21.15.42:0/2074875051 >> 172.21.15.42:0/2074875051 conn(0x7f42d00fba20 msgr2=0x7f42d00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 -- 172.21.15.42:0/2074875051 shutdown_connections 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 -- 172.21.15.42:0/2074875051 wait complete. 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 Processor -- start 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.555+0000 7f42d693d700 1 -- start start 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42d693d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 0x7f42d0196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42d693d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42d01016a0 0x7f42d01969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42d693d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d0102040 0x7f42d019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42d693d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d019df90 con 0x7f42d01004b0 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42d693d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d019e0d0 con 0x7f42d0102040 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42d693d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d019e260 con 0x7f42d01016a0 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 0x7f42d0196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 0x7f42d0196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33634/0 (socket says 172.21.15.42:33634) 2023-12-12T09:56:50.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cffff700 1 -- 172.21.15.42:0/2487806199 learned_addr learned my addr 172.21.15.42:0/2487806199 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42d4eda700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d0102040 0x7f42d019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cf7fe700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42d01016a0 0x7f42d01969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cffff700 1 -- 172.21.15.42:0/2487806199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42d01016a0 msgr2=0x7f42d01969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cffff700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42d01016a0 0x7f42d01969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cffff700 1 -- 172.21.15.42:0/2487806199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d0102040 msgr2=0x7f42d019d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cffff700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d0102040 0x7f42d019d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cffff700 1 -- 172.21.15.42:0/2487806199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c00095f0 con 0x7f42d01004b0 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.556+0000 7f42cffff700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 0x7f42d0196460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009f50 tx=0x7f42c00047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.557+0000 7f42cd7fa700 1 -- 172.21.15.42:0/2487806199 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c001f070 con 0x7f42d01004b0 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.557+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d00697b0 con 0x7f42d01004b0 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.557+0000 7f42cd7fa700 1 -- 172.21.15.42:0/2487806199 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42c001ac50 con 0x7f42d01004b0 2023-12-12T09:56:50.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.557+0000 7f42cd7fa700 1 -- 172.21.15.42:0/2487806199 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c00041c0 con 0x7f42d01004b0 2023-12-12T09:56:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.557+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d0069c60 con 0x7f42d01004b0 2023-12-12T09:56:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.558+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d0067a80 con 0x7f42d01004b0 2023-12-12T09:56:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.559+0000 7f42cd7fa700 1 -- 172.21.15.42:0/2487806199 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f42c0023070 con 0x7f42d01004b0 2023-12-12T09:56:50.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.559+0000 7f42cd7fa700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f42bc0a1480 0x7f42bc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.559+0000 7f42cd7fa700 1 -- 172.21.15.42:0/2487806199 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f42c00d0510 con 0x7f42d01004b0 2023-12-12T09:56:50.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.559+0000 7f42cf7fe700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f42bc0a1480 0x7f42bc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.560+0000 7f42cf7fe700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f42bc0a1480 0x7f42bc0a3940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005950 tx=0x7f42b800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:50.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.563+0000 7f42cd7fa700 1 -- 172.21.15.42:0/2487806199 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f42c0099a10 con 0x7f42d01004b0 2023-12-12T09:56:50.764 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.762+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42d010bef0 con 0x7f42d01004b0 2023-12-12T09:56:50.765 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.763+0000 7f42cd7fa700 1 -- 172.21.15.42:0/2487806199 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42c009d030 con 0x7f42d01004b0 2023-12-12T09:56:50.765 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T09:56:50.767 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f42bc0a1480 msgr2=0x7f42bc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.767 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f42bc0a1480 0x7f42bc0a3940 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005950 tx=0x7f42b800a300 comp rx=0 tx=0).stop 2023-12-12T09:56:50.767 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 msgr2=0x7f42d0196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.767 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 0x7f42d0196460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009f50 tx=0x7f42c00047a0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.767 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 shutdown_connections 2023-12-12T09:56:50.767 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f42bc0a1480 0x7f42bc0a3940 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.767 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42d0102040 0x7f42d019d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.767 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42d01004b0 0x7f42d0196460 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.768 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 --2- 172.21.15.42:0/2487806199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42d01016a0 0x7f42d01969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.768 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 >> 172.21.15.42:0/2487806199 conn(0x7f42d00fba20 msgr2=0x7f42d0104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:50.768 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 shutdown_connections 2023-12-12T09:56:50.768 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.765+0000 7f42d693d700 1 -- 172.21.15.42:0/2487806199 wait complete. 2023-12-12T09:56:50.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:50 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2415861407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:50.779 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:56:50.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:50 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2415861407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:50.910 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.905+0000 7f986484a700 1 -- 172.21.15.42:0/1976754919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860102010 msgr2=0x7f98601088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.905+0000 7f986484a700 1 --2- 172.21.15.42:0/1976754919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860102010 0x7f98601088c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f985800b0d0 tx=0x7f985800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.905+0000 7f986484a700 1 -- 172.21.15.42:0/1976754919 shutdown_connections 2023-12-12T09:56:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.905+0000 7f986484a700 1 --2- 172.21.15.42:0/1976754919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860102010 0x7f98601088c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.905+0000 7f986484a700 1 --2- 172.21.15.42:0/1976754919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9860100480 0x7f98601008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.905+0000 7f986484a700 1 --2- 172.21.15.42:0/1976754919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9860101670 0x7f9860101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 -- 172.21.15.42:0/1976754919 >> 172.21.15.42:0/1976754919 conn(0x7f98600fb9f0 msgr2=0x7f98600fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:50.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 -- 172.21.15.42:0/1976754919 shutdown_connections 2023-12-12T09:56:50.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 -- 172.21.15.42:0/1976754919 wait complete. 2023-12-12T09:56:50.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 Processor -- start 2023-12-12T09:56:50.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 -- start start 2023-12-12T09:56:50.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9860100480 0x7f98601043c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860101670 0x7f9860102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9860102010 0x7f9860102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860103730 con 0x7f9860101670 2023-12-12T09:56:50.913 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98601038a0 con 0x7f9860102010 2023-12-12T09:56:50.913 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.906+0000 7f986484a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98601a1660 con 0x7f9860100480 2023-12-12T09:56:50.913 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860101670 0x7f9860102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.913 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860101670 0x7f9860102a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33652/0 (socket says 172.21.15.42:33652) 2023-12-12T09:56:50.913 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985dd9b700 1 -- 172.21.15.42:0/343497023 learned_addr learned my addr 172.21.15.42:0/343497023 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:50.913 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985ed9d700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9860102010 0x7f9860102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.913 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985e59c700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9860100480 0x7f98601043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985dd9b700 1 -- 172.21.15.42:0/343497023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9860100480 msgr2=0x7f98601043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985dd9b700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9860100480 0x7f98601043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985dd9b700 1 -- 172.21.15.42:0/343497023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9860102010 msgr2=0x7f9860102f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:50.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985dd9b700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9860102010 0x7f9860102f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:50.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985dd9b700 1 -- 172.21.15.42:0/343497023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858009d00 con 0x7f9860101670 2023-12-12T09:56:50.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985e59c700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9860100480 0x7f98601043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:56:50.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f985dd9b700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860101670 0x7f9860102a50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f985400d950 tx=0x7f985400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:50.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f984f7fe700 1 -- 172.21.15.42:0/343497023 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98540098e0 con 0x7f9860101670 2023-12-12T09:56:50.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98601a18f0 con 0x7f9860101670 2023-12-12T09:56:50.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f984f7fe700 1 -- 172.21.15.42:0/343497023 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9854010460 con 0x7f9860101670 2023-12-12T09:56:50.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f984f7fe700 1 -- 172.21.15.42:0/343497023 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985400f620 con 0x7f9860101670 2023-12-12T09:56:50.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.907+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98601a1e00 con 0x7f9860101670 2023-12-12T09:56:50.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.908+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9860068310 con 0x7f9860101670 2023-12-12T09:56:50.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.909+0000 7f984f7fe700 1 -- 172.21.15.42:0/343497023 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f98540108d0 con 0x7f9860101670 2023-12-12T09:56:50.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.910+0000 7f984f7fe700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98480a1480 0x7f98480a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:50.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.910+0000 7f984f7fe700 1 -- 172.21.15.42:0/343497023 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f98540ccd60 con 0x7f9860101670 2023-12-12T09:56:50.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.910+0000 7f985e59c700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98480a1480 0x7f98480a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:50.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.911+0000 7f985e59c700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98480a1480 0x7f98480a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f98601014d0 tx=0x7f9850005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:50.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:50.914+0000 7f984f7fe700 1 -- 172.21.15.42:0/343497023 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f98540961d0 con 0x7f9860101670 2023-12-12T09:56:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:50 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2415861407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:51.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.114+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98601a2060 con 0x7f9860101670 2023-12-12T09:56:51.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.114+0000 7f984f7fe700 1 -- 172.21.15.42:0/343497023 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9854016020 con 0x7f9860101670 2023-12-12T09:56:51.116 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98480a1480 msgr2=0x7f98480a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98480a1480 0x7f98480a3940 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f98601014d0 tx=0x7f9850005f90 comp rx=0 tx=0).stop 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860101670 msgr2=0x7f9860102a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860101670 0x7f9860102a50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f985400d950 tx=0x7f985400dc60 comp rx=0 tx=0).stop 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 shutdown_connections 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98480a1480 0x7f98480a3940 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9860102010 0x7f9860102f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9860100480 0x7f98601043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 --2- 172.21.15.42:0/343497023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9860101670 0x7f9860102a50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 >> 172.21.15.42:0/343497023 conn(0x7f98600fb9f0 msgr2=0x7f98600fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.116+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 shutdown_connections 2023-12-12T09:56:51.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.117+0000 7f986484a700 1 -- 172.21.15.42:0/343497023 wait complete. 2023-12-12T09:56:51.130 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:56:51.261 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.257+0000 7f153abfb700 1 -- 172.21.15.42:0/985644290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534100480 msgr2=0x7f15341008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.261 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.257+0000 7f153abfb700 1 --2- 172.21.15.42:0/985644290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534100480 0x7f15341008a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f15280099c0 tx=0x7f1528009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.257+0000 7f153abfb700 1 -- 172.21.15.42:0/985644290 shutdown_connections 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.257+0000 7f153abfb700 1 --2- 172.21.15.42:0/985644290 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1534102010 0x7f15341088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.257+0000 7f153abfb700 1 --2- 172.21.15.42:0/985644290 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534100480 0x7f15341008a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.257+0000 7f153abfb700 1 --2- 172.21.15.42:0/985644290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1534101670 0x7f1534101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.257+0000 7f153abfb700 1 -- 172.21.15.42:0/985644290 >> 172.21.15.42:0/985644290 conn(0x7f15340fb9f0 msgr2=0x7f15340fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.257+0000 7f153abfb700 1 -- 172.21.15.42:0/985644290 shutdown_connections 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.257+0000 7f153abfb700 1 -- 172.21.15.42:0/985644290 wait complete. 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f153abfb700 1 Processor -- start 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f153abfb700 1 -- start start 2023-12-12T09:56:51.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f153abfb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1534100480 0x7f1534197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f153abfb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1534101670 0x7f1534197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f153abfb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534102010 0x7f153419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f153abfb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153419f040 con 0x7f1534102010 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f153abfb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153419f180 con 0x7f1534100480 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f153abfb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153419f310 con 0x7f1534101670 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1539198700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534102010 0x7f153419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1539198700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534102010 0x7f153419ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33666/0 (socket says 172.21.15.42:33666) 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1539198700 1 -- 172.21.15.42:0/2106820913 learned_addr learned my addr 172.21.15.42:0/2106820913 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1538997700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1534100480 0x7f1534197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1533fff700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1534101670 0x7f1534197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1539198700 1 -- 172.21.15.42:0/2106820913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1534101670 msgr2=0x7f1534197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1539198700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1534101670 0x7f1534197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.263 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1539198700 1 -- 172.21.15.42:0/2106820913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1534100480 msgr2=0x7f1534197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1539198700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1534100480 0x7f1534197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.258+0000 7f1539198700 1 -- 172.21.15.42:0/2106820913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15280095f0 con 0x7f1534102010 2023-12-12T09:56:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.259+0000 7f1539198700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534102010 0x7f153419ea70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f152c00ef30 tx=0x7f152c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.259+0000 7f1531ffb700 1 -- 172.21.15.42:0/2106820913 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c018070 con 0x7f1534102010 2023-12-12T09:56:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.259+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153419f590 con 0x7f1534102010 2023-12-12T09:56:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.259+0000 7f1531ffb700 1 -- 172.21.15.42:0/2106820913 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f152c00f040 con 0x7f1534102010 2023-12-12T09:56:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.259+0000 7f1531ffb700 1 -- 172.21.15.42:0/2106820913 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c00cdb0 con 0x7f1534102010 2023-12-12T09:56:51.264 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.259+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153419fa50 con 0x7f1534102010 2023-12-12T09:56:51.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.260+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15340682f0 con 0x7f1534102010 2023-12-12T09:56:51.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.261+0000 7f1531ffb700 1 -- 172.21.15.42:0/2106820913 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f152c003a00 con 0x7f1534102010 2023-12-12T09:56:51.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.262+0000 7f1531ffb700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f15240a1480 0x7f15240a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.262+0000 7f1531ffb700 1 -- 172.21.15.42:0/2106820913 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f152c0cd4d0 con 0x7f1534102010 2023-12-12T09:56:51.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.262+0000 7f1538997700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f15240a1480 0x7f15240a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.262+0000 7f1538997700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f15240a1480 0x7f15240a3940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1528003730 tx=0x7f1528000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:51.267 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.265+0000 7f1531ffb700 1 -- 172.21.15.42:0/2106820913 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f152c0969d0 con 0x7f1534102010 2023-12-12T09:56:51.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.467+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f153410bca0 con 0x7f1534102010 2023-12-12T09:56:51.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.467+0000 7f1531ffb700 1 -- 172.21.15.42:0/2106820913 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f152c099ff0 con 0x7f1534102010 2023-12-12T09:56:51.470 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2023-12-12T09:56:51.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f15240a1480 msgr2=0x7f15240a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f15240a1480 0x7f15240a3940 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1528003730 tx=0x7f1528000bc0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534102010 msgr2=0x7f153419ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534102010 0x7f153419ea70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f152c00ef30 tx=0x7f152c00c3a0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 shutdown_connections 2023-12-12T09:56:51.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f15240a1480 0x7f15240a3940 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1534100480 0x7f1534197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1534102010 0x7f153419ea70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 --2- 172.21.15.42:0/2106820913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1534101670 0x7f1534197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 >> 172.21.15.42:0/2106820913 conn(0x7f15340fb9f0 msgr2=0x7f1534068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:51.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 shutdown_connections 2023-12-12T09:56:51.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.470+0000 7f153abfb700 1 -- 172.21.15.42:0/2106820913 wait complete. 2023-12-12T09:56:51.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.610+0000 7fe4c112f700 1 -- 172.21.15.42:0/2468307504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc103130 msgr2=0x7fe4bc1099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.610+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2468307504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc103130 0x7fe4bc1099e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400b0d0 tx=0x7fe4b400b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.610+0000 7fe4c112f700 1 -- 172.21.15.42:0/2468307504 shutdown_connections 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.610+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2468307504 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc103130 0x7fe4bc1099e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.610+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2468307504 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4bc1015a0 0x7fe4bc1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.610+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2468307504 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4bc102790 0x7fe4bc102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.610+0000 7fe4c112f700 1 -- 172.21.15.42:0/2468307504 >> 172.21.15.42:0/2468307504 conn(0x7fe4bc0fcb10 msgr2=0x7fe4bc0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.610+0000 7fe4c112f700 1 -- 172.21.15.42:0/2468307504 shutdown_connections 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.610+0000 7fe4c112f700 1 -- 172.21.15.42:0/2468307504 wait complete. 2023-12-12T09:56:51.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4c112f700 1 Processor -- start 2023-12-12T09:56:51.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4c112f700 1 -- start start 2023-12-12T09:56:51.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4c112f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc1015a0 0x7fe4bc197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4c112f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4bc102790 0x7fe4bc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4c112f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4bc103130 0x7fe4bc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4c112f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc19f130 con 0x7fe4bc1015a0 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4c112f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc19f270 con 0x7fe4bc103130 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4c112f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4bc19f400 con 0x7fe4bc102790 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4bad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc1015a0 0x7fe4bc197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.611+0000 7fe4bad9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc1015a0 0x7fe4bc197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33676/0 (socket says 172.21.15.42:33676) 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4bad9d700 1 -- 172.21.15.42:0/2797540561 learned_addr learned my addr 172.21.15.42:0/2797540561 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4bb59e700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4bc103130 0x7fe4bc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4ba59c700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4bc102790 0x7fe4bc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4bad9d700 1 -- 172.21.15.42:0/2797540561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4bc102790 msgr2=0x7fe4bc197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4bad9d700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4bc102790 0x7fe4bc197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4bad9d700 1 -- 172.21.15.42:0/2797540561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4bc103130 msgr2=0x7fe4bc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4bad9d700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4bc103130 0x7fe4bc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4bad9d700 1 -- 172.21.15.42:0/2797540561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4009d00 con 0x7fe4bc1015a0 2023-12-12T09:56:51.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4bad9d700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc1015a0 0x7fe4bc197600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00e970 tx=0x7fe4ac00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:51.617 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4a3fff700 1 -- 172.21.15.42:0/2797540561 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac009790 con 0x7fe4bc1015a0 2023-12-12T09:56:51.617 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4bc19f6e0 con 0x7fe4bc1015a0 2023-12-12T09:56:51.617 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4a3fff700 1 -- 172.21.15.42:0/2797540561 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4ac00c7e0 con 0x7fe4bc1015a0 2023-12-12T09:56:51.617 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4a3fff700 1 -- 172.21.15.42:0/2797540561 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac003c70 con 0x7fe4bc1015a0 2023-12-12T09:56:51.617 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.612+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4bc19fc40 con 0x7fe4bc1015a0 2023-12-12T09:56:51.617 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.613+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4bc108950 con 0x7fe4bc1015a0 2023-12-12T09:56:51.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.614+0000 7fe4a3fff700 1 -- 172.21.15.42:0/2797540561 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fe4ac00cc50 con 0x7fe4bc1015a0 2023-12-12T09:56:51.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.615+0000 7fe4a3fff700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe4a40a1480 0x7fe4a40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.615+0000 7fe4a3fff700 1 -- 172.21.15.42:0/2797540561 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe4ac014070 con 0x7fe4bc1015a0 2023-12-12T09:56:51.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.615+0000 7fe4ba59c700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe4a40a1480 0x7fe4a40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.615+0000 7fe4ba59c700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe4a40a1480 0x7fe4a40a3940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0005950 tx=0x7fe4b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:51.621 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.619+0000 7fe4a3fff700 1 -- 172.21.15.42:0/2797540561 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe4ac096530 con 0x7fe4bc1015a0 2023-12-12T09:56:51.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.820+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe4bc10cff0 con 0x7fe4bc1015a0 2023-12-12T09:56:51.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.820+0000 7fe4a3fff700 1 -- 172.21.15.42:0/2797540561 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe4ac099b50 con 0x7fe4bc1015a0 2023-12-12T09:56:51.822 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:56:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:51 smithi173 ceph-mon[103308]: pgmap v268: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2023-12-12T09:56:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:51 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2487806199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:51 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/343497023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:56:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:51 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2106820913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:56:51.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe4a40a1480 msgr2=0x7fe4a40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe4a40a1480 0x7fe4a40a3940 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0005950 tx=0x7fe4b000b410 comp rx=0 tx=0).stop 2023-12-12T09:56:51.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc1015a0 msgr2=0x7fe4bc197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc1015a0 0x7fe4bc197600 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00e970 tx=0x7fe4ac00ec80 comp rx=0 tx=0).stop 2023-12-12T09:56:51.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 shutdown_connections 2023-12-12T09:56:51.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe4a40a1480 0x7fe4a40a3940 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4bc103130 0x7fe4bc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4bc1015a0 0x7fe4bc197600 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 --2- 172.21.15.42:0/2797540561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4bc102790 0x7fe4bc197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 >> 172.21.15.42:0/2797540561 conn(0x7fe4bc0fcb10 msgr2=0x7fe4bc105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:51.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 shutdown_connections 2023-12-12T09:56:51.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.822+0000 7fe4c112f700 1 -- 172.21.15.42:0/2797540561 wait complete. 2023-12-12T09:56:51.835 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:51 smithi042 ceph-mon[98894]: pgmap v268: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2023-12-12T09:56:51.835 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:51 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2487806199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:51.835 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:51 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/343497023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:56:51.835 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:51 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2106820913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:56:51.836 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:56:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.961+0000 7fd1d8586700 1 -- 172.21.15.42:0/2693605228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0102010 msgr2=0x7fd1d01088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.961+0000 7fd1d8586700 1 --2- 172.21.15.42:0/2693605228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0102010 0x7fd1d01088c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800b0d0 tx=0x7fd1c800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.961+0000 7fd1d8586700 1 -- 172.21.15.42:0/2693605228 shutdown_connections 2023-12-12T09:56:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.961+0000 7fd1d8586700 1 --2- 172.21.15.42:0/2693605228 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d0100480 0x7fd1d01008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.961+0000 7fd1d8586700 1 --2- 172.21.15.42:0/2693605228 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0102010 0x7fd1d01088c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.961+0000 7fd1d8586700 1 --2- 172.21.15.42:0/2693605228 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1d0101670 0x7fd1d0101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.961+0000 7fd1d8586700 1 -- 172.21.15.42:0/2693605228 >> 172.21.15.42:0/2693605228 conn(0x7fd1d00fb9f0 msgr2=0x7fd1d00fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:51.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.961+0000 7fd1d8586700 1 -- 172.21.15.42:0/2693605228 shutdown_connections 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.961+0000 7fd1d8586700 1 -- 172.21.15.42:0/2693605228 wait complete. 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d8586700 1 Processor -- start 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d8586700 1 -- start start 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d8586700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0100480 0x7fd1d0069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d8586700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d0101670 0x7fd1d019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d8586700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1d0102010 0x7fd1d019e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d8586700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d019f080 con 0x7fd1d0100480 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d8586700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d019f1f0 con 0x7fd1d0101670 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d8586700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d019f3b0 con 0x7fd1d0102010 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6322700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0100480 0x7fd1d0069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6322700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0100480 0x7fd1d0069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33690/0 (socket says 172.21.15.42:33690) 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6322700 1 -- 172.21.15.42:0/1793333765 learned_addr learned my addr 172.21.15.42:0/1793333765 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d5b21700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d0101670 0x7fd1d019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6b23700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1d0102010 0x7fd1d019e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6322700 1 -- 172.21.15.42:0/1793333765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1d0102010 msgr2=0x7fd1d019e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6322700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1d0102010 0x7fd1d019e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6322700 1 -- 172.21.15.42:0/1793333765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d0101670 msgr2=0x7fd1d019c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6322700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d0101670 0x7fd1d019c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6322700 1 -- 172.21.15.42:0/1793333765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c8009d00 con 0x7fd1d0100480 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d5b21700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d0101670 0x7fd1d019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.962+0000 7fd1d6322700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0100480 0x7fd1d0069130 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000c990 tx=0x7fd1c000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.963+0000 7fd1c77fe700 1 -- 172.21.15.42:0/1793333765 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0007b00 con 0x7fd1d0100480 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.963+0000 7fd1c77fe700 1 -- 172.21.15.42:0/1793333765 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1c0007c60 con 0x7fd1d0100480 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.963+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d019f690 con 0x7fd1d0100480 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.963+0000 7fd1c77fe700 1 -- 172.21.15.42:0/1793333765 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c0018660 con 0x7fd1d0100480 2023-12-12T09:56:51.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.963+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d019fba0 con 0x7fd1d0100480 2023-12-12T09:56:51.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.963+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d01077d0 con 0x7fd1d0100480 2023-12-12T09:56:51.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.965+0000 7fd1c77fe700 1 -- 172.21.15.42:0/1793333765 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd1c0004020 con 0x7fd1d0100480 2023-12-12T09:56:51.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.966+0000 7fd1c77fe700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd1bc0a1480 0x7fd1bc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:51.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.966+0000 7fd1d5b21700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd1bc0a1480 0x7fd1bc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:51.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.966+0000 7fd1c77fe700 1 -- 172.21.15.42:0/1793333765 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd1c00cd000 con 0x7fd1d0100480 2023-12-12T09:56:51.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.966+0000 7fd1d5b21700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd1bc0a1480 0x7fd1bc0a3940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00696f0 tx=0x7fd1cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:51.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:51.969+0000 7fd1c77fe700 1 -- 172.21.15.42:0/1793333765 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd1c0096500 con 0x7fd1d0100480 2023-12-12T09:56:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:51 smithi111 ceph-mon[102565]: pgmap v268: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 383 B/s wr, 3 op/s 2023-12-12T09:56:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:51 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2487806199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:51 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/343497023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:56:51.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:51 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2106820913' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:56:52.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.191+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd1d0068310 con 0x7fd1d0100480 2023-12-12T09:56:52.193 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.191+0000 7fd1c77fe700 1 -- 172.21.15.42:0/1793333765 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+591 (secure 0 0 0) 0x7fd1c0099b20 con 0x7fd1d0100480 2023-12-12T09:56:52.193 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/467026676 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:52.193 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2920147377 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:52.193 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/1525180467 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:52.194 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/1433305616 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:52.194 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/1433305616 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:52.194 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3856214912 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:52.194 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/1525180467 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:52.194 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1613862950 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:52.194 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4273364865 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:52.194 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2212533654 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:52.195 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd1bc0a1480 msgr2=0x7fd1bc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd1bc0a1480 0x7fd1bc0a3940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00696f0 tx=0x7fd1cc009260 comp rx=0 tx=0).stop 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0100480 msgr2=0x7fd1d0069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0100480 0x7fd1d0069130 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000c990 tx=0x7fd1c000cd50 comp rx=0 tx=0).stop 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 shutdown_connections 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd1bc0a1480 0x7fd1bc0a3940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1d0102010 0x7fd1d019e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1d0100480 0x7fd1d0069130 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 --2- 172.21.15.42:0/1793333765 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd1d0101670 0x7fd1d019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.193+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 >> 172.21.15.42:0/1793333765 conn(0x7fd1d00fb9f0 msgr2=0x7fd1d0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.194+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 shutdown_connections 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.194+0000 7fd1d8586700 1 -- 172.21.15.42:0/1793333765 wait complete. 2023-12-12T09:56:52.196 INFO:teuthology.orchestra.run.smithi042.stderr:listed 10 entries 2023-12-12T09:56:52.207 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:56:52.208 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:56:52.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:56:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 -- 172.21.15.42:0/2133321229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb874101f40 msgr2=0x7fb8741087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/2133321229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb874101f40 0x7fb8741087f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00b0d0 tx=0x7fb86c00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 -- 172.21.15.42:0/2133321229 shutdown_connections 2023-12-12T09:56:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/2133321229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb874101f40 0x7fb8741087f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/2133321229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8741003b0 0x7fb8741007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/2133321229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb8741015a0 0x7fb874101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 -- 172.21.15.42:0/2133321229 >> 172.21.15.42:0/2133321229 conn(0x7fb8740fb920 msgr2=0x7fb8740fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 -- 172.21.15.42:0/2133321229 shutdown_connections 2023-12-12T09:56:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 -- 172.21.15.42:0/2133321229 wait complete. 2023-12-12T09:56:52.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 Processor -- start 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.390+0000 7fb87a4f6700 1 -- start start 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb87a4f6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8741003b0 0x7fb874196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb87a4f6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8741015a0 0x7fb874196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb87a4f6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb874101f40 0x7fb87419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb87a4f6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87419df30 con 0x7fb8741003b0 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb87a4f6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87419e070 con 0x7fb874101f40 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb87a4f6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87419e200 con 0x7fb8741015a0 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb873fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8741003b0 0x7fb874196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb873fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8741003b0 0x7fb874196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33704/0 (socket says 172.21.15.42:33704) 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb873fff700 1 -- 172.21.15.42:0/156419742 learned_addr learned my addr 172.21.15.42:0/156419742 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb878a93700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb874101f40 0x7fb87419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb8737fe700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8741015a0 0x7fb874196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:52.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb873fff700 1 -- 172.21.15.42:0/156419742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8741015a0 msgr2=0x7fb874196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb873fff700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8741015a0 0x7fb874196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb873fff700 1 -- 172.21.15.42:0/156419742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb874101f40 msgr2=0x7fb87419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb873fff700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb874101f40 0x7fb87419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb873fff700 1 -- 172.21.15.42:0/156419742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c009d00 con 0x7fb8741003b0 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb873fff700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8741003b0 0x7fb874196400 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb86400d980 tx=0x7fb86400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb8717fa700 1 -- 172.21.15.42:0/156419742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb864009930 con 0x7fb8741003b0 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb8717fa700 1 -- 172.21.15.42:0/156419742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb864010460 con 0x7fb8741003b0 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87419e4e0 con 0x7fb8741003b0 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.391+0000 7fb8717fa700 1 -- 172.21.15.42:0/156419742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86400f620 con 0x7fb8741003b0 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.392+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87419e9a0 con 0x7fb8741003b0 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.392+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb874107780 con 0x7fb8741003b0 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.393+0000 7fb8717fa700 1 -- 172.21.15.42:0/156419742 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fb864010860 con 0x7fb8741003b0 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.394+0000 7fb8717fa700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb85c0a1480 0x7fb85c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:52.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.394+0000 7fb8717fa700 1 -- 172.21.15.42:0/156419742 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb8640cc9d0 con 0x7fb8741003b0 2023-12-12T09:56:52.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.394+0000 7fb8737fe700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb85c0a1480 0x7fb85c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:52.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.395+0000 7fb8737fe700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb85c0a1480 0x7fb85c0a3940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb868005fd0 tx=0x7fb868005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:52.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.398+0000 7fb8717fa700 1 -- 172.21.15.42:0/156419742 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb864095e40 con 0x7fb8741003b0 2023-12-12T09:56:52.605 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.603+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb87410bbe0 con 0x7fb8741003b0 2023-12-12T09:56:52.606 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.604+0000 7fb8717fa700 1 -- 172.21.15.42:0/156419742 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7fb864020090 con 0x7fb8741003b0 2023-12-12T09:56:52.607 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:56:52.607 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:56:52.609 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb85c0a1480 msgr2=0x7fb85c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.609 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb85c0a1480 0x7fb85c0a3940 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb868005fd0 tx=0x7fb868005f60 comp rx=0 tx=0).stop 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8741003b0 msgr2=0x7fb874196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8741003b0 0x7fb874196400 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb86400d980 tx=0x7fb86400dc90 comp rx=0 tx=0).stop 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 shutdown_connections 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb85c0a1480 0x7fb85c0a3940 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb874101f40 0x7fb87419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb8741003b0 0x7fb874196400 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 --2- 172.21.15.42:0/156419742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8741015a0 0x7fb874196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 >> 172.21.15.42:0/156419742 conn(0x7fb8740fb920 msgr2=0x7fb8741048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 shutdown_connections 2023-12-12T09:56:52.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.607+0000 7fb87a4f6700 1 -- 172.21.15.42:0/156419742 wait complete. 2023-12-12T09:56:52.622 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:56:35.884237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:56:37.283813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:56:40.918865+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:56:52.623 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:56:52.754 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.749+0000 7f39c55e0700 1 -- 172.21.15.42:0/1958989753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 msgr2=0x7f39c0101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.754 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.749+0000 7f39c55e0700 1 --2- 172.21.15.42:0/1958989753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 0x7f39c0101ae0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f39a80099c0 tx=0x7f39a8009cd0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.754 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.749+0000 7f39c55e0700 1 -- 172.21.15.42:0/1958989753 shutdown_connections 2023-12-12T09:56:52.754 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.749+0000 7f39c55e0700 1 --2- 172.21.15.42:0/1958989753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39c0100490 0x7f39c01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.754 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.749+0000 7f39c55e0700 1 --2- 172.21.15.42:0/1958989753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c0102020 0x7f39c01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.754 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.749+0000 7f39c55e0700 1 --2- 172.21.15.42:0/1958989753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 0x7f39c0101ae0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.754 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.749+0000 7f39c55e0700 1 -- 172.21.15.42:0/1958989753 >> 172.21.15.42:0/1958989753 conn(0x7f39c00fba20 msgr2=0x7f39c00fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.749+0000 7f39c55e0700 1 -- 172.21.15.42:0/1958989753 shutdown_connections 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.749+0000 7f39c55e0700 1 -- 172.21.15.42:0/1958989753 wait complete. 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39c55e0700 1 Processor -- start 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39c55e0700 1 -- start start 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39c55e0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39c0100490 0x7f39c0197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39c55e0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 0x7f39c0197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39c55e0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c0102020 0x7f39c019ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39c55e0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c019f0a0 con 0x7f39c0101680 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39c55e0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c019f1e0 con 0x7f39c0100490 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39c55e0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c019f370 con 0x7f39c0102020 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39be7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 0x7f39c0197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39be7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 0x7f39c0197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33720/0 (socket says 172.21.15.42:33720) 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.750+0000 7f39be7fc700 1 -- 172.21.15.42:0/2631779004 learned_addr learned my addr 172.21.15.42:0/2631779004 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:52.755 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39beffd700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39c0100490 0x7f39c0197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39bf7fe700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c0102020 0x7f39c019ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39be7fc700 1 -- 172.21.15.42:0/2631779004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c0102020 msgr2=0x7f39c019ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39be7fc700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c0102020 0x7f39c019ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39be7fc700 1 -- 172.21.15.42:0/2631779004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39c0100490 msgr2=0x7f39c0197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39be7fc700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39c0100490 0x7f39c0197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39be7fc700 1 -- 172.21.15.42:0/2631779004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80095f0 con 0x7f39c0101680 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39be7fc700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 0x7f39c0197ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f39a8003730 tx=0x7f39a800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39b7fff700 1 -- 172.21.15.42:0/2631779004 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a801d070 con 0x7f39c0101680 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39b7fff700 1 -- 172.21.15.42:0/2631779004 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a800fe70 con 0x7f39c0101680 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c019f5f0 con 0x7f39c0101680 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39b7fff700 1 -- 172.21.15.42:0/2631779004 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8017650 con 0x7f39c0101680 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.751+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c019fa10 con 0x7f39c0101680 2023-12-12T09:56:52.756 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.752+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c0068760 con 0x7f39c0101680 2023-12-12T09:56:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.753+0000 7f39b7fff700 1 -- 172.21.15.42:0/2631779004 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f39a8022470 con 0x7f39c0101680 2023-12-12T09:56:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.754+0000 7f39b7fff700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39ac0a1480 0x7f39ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.754+0000 7f39beffd700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39ac0a1480 0x7f39ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.754+0000 7f39b7fff700 1 -- 172.21.15.42:0/2631779004 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f39a80ce630 con 0x7f39c0101680 2023-12-12T09:56:52.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.755+0000 7f39beffd700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39ac0a1480 0x7f39ac0a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f39c01014e0 tx=0x7f39b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:52.759 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.758+0000 7f39b7fff700 1 -- 172.21.15.42:0/2631779004 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f39a8097ad0 con 0x7f39c0101680 2023-12-12T09:56:52.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:52 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2797540561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:56:52.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:52 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1793333765' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-12T09:56:52.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:52 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2797540561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:56:52.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:52 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1793333765' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-12T09:56:52.961 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.959+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39c019fcd0 con 0x7f39c0101680 2023-12-12T09:56:52.961 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.959+0000 7f39b7fff700 1 -- 172.21.15.42:0/2631779004 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39a809b0f0 con 0x7f39c0101680 2023-12-12T09:56:52.962 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T09:56:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39ac0a1480 msgr2=0x7f39ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39ac0a1480 0x7f39ac0a3940 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f39c01014e0 tx=0x7f39b0009260 comp rx=0 tx=0).stop 2023-12-12T09:56:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 msgr2=0x7f39c0197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 0x7f39c0197ab0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f39a8003730 tx=0x7f39a800fa00 comp rx=0 tx=0).stop 2023-12-12T09:56:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 shutdown_connections 2023-12-12T09:56:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39ac0a1480 0x7f39ac0a3940 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39c0100490 0x7f39c0197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39c0102020 0x7f39c019ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 --2- 172.21.15.42:0/2631779004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39c0101680 0x7f39c0197ab0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:52.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 >> 172.21.15.42:0/2631779004 conn(0x7f39c00fba20 msgr2=0x7f39c00fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:52.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 shutdown_connections 2023-12-12T09:56:52.965 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:52.962+0000 7f39c55e0700 1 -- 172.21.15.42:0/2631779004 wait complete. 2023-12-12T09:56:52.976 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:56:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:52 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2797540561' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:56:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:52 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1793333765' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-12T09:56:53.105 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.102+0000 7f39688c9700 1 -- 172.21.15.42:0/545341731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 msgr2=0x7f3964106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.105 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.102+0000 7f39688c9700 1 --2- 172.21.15.42:0/545341731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 0x7f3964106d20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f394c009990 tx=0x7f394c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.105 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.102+0000 7f39688c9700 1 -- 172.21.15.42:0/545341731 shutdown_connections 2023-12-12T09:56:53.105 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.102+0000 7f39688c9700 1 --2- 172.21.15.42:0/545341731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3964107260 0x7f39641096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.105 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.102+0000 7f39688c9700 1 --2- 172.21.15.42:0/545341731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3964102000 0x7f39641043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.105 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.102+0000 7f39688c9700 1 --2- 172.21.15.42:0/545341731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 0x7f3964106d20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.105 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.102+0000 7f39688c9700 1 -- 172.21.15.42:0/545341731 >> 172.21.15.42:0/545341731 conn(0x7f39640fb9f0 msgr2=0x7f39640fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:53.105 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.102+0000 7f39688c9700 1 -- 172.21.15.42:0/545341731 shutdown_connections 2023-12-12T09:56:53.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.102+0000 7f39688c9700 1 -- 172.21.15.42:0/545341731 wait complete. 2023-12-12T09:56:53.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.103+0000 7f39688c9700 1 Processor -- start 2023-12-12T09:56:53.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.103+0000 7f39688c9700 1 -- start start 2023-12-12T09:56:53.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.103+0000 7f39688c9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3964102000 0x7f39641974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.103+0000 7f39688c9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 0x7f3964197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.103+0000 7f39688c9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3964107260 0x7f396419ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.103+0000 7f39688c9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396419f020 con 0x7f3964104930 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.103+0000 7f39688c9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396419f160 con 0x7f3964102000 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.103+0000 7f39688c9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396419f2f0 con 0x7f3964107260 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3961d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 0x7f3964197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3961d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 0x7f3964197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33734/0 (socket says 172.21.15.42:33734) 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3961d9b700 1 -- 172.21.15.42:0/3598184417 learned_addr learned my addr 172.21.15.42:0/3598184417 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3962d9d700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3964107260 0x7f396419ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f396259c700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3964102000 0x7f39641974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3961d9b700 1 -- 172.21.15.42:0/3598184417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3964107260 msgr2=0x7f396419ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3961d9b700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3964107260 0x7f396419ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3961d9b700 1 -- 172.21.15.42:0/3598184417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3964102000 msgr2=0x7f39641974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3961d9b700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3964102000 0x7f39641974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3961d9b700 1 -- 172.21.15.42:0/3598184417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c0095f0 con 0x7f3964104930 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f3961d9b700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 0x7f3964197a30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f394c000c00 tx=0x7f394c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:53.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f395b7fe700 1 -- 172.21.15.42:0/3598184417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c01d070 con 0x7f3964104930 2023-12-12T09:56:53.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396419f570 con 0x7f3964104930 2023-12-12T09:56:53.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f395b7fe700 1 -- 172.21.15.42:0/3598184417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f394c0054f0 con 0x7f3964104930 2023-12-12T09:56:53.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f395b7fe700 1 -- 172.21.15.42:0/3598184417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c004bd0 con 0x7f3964104930 2023-12-12T09:56:53.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.104+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396419f990 con 0x7f3964104930 2023-12-12T09:56:53.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.105+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39640fd570 con 0x7f3964104930 2023-12-12T09:56:53.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.106+0000 7f395b7fe700 1 -- 172.21.15.42:0/3598184417 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f394c00fb90 con 0x7f3964104930 2023-12-12T09:56:53.110 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.107+0000 7f395b7fe700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39500a1480 0x7f39500a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.110 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.107+0000 7f396259c700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39500a1480 0x7f39500a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.110 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.107+0000 7f395b7fe700 1 -- 172.21.15.42:0/3598184417 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f394c0ce270 con 0x7f3964104930 2023-12-12T09:56:53.110 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.108+0000 7f396259c700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39500a1480 0x7f39500a3940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3954009c60 tx=0x7f3954009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:53.112 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.110+0000 7f395b7fe700 1 -- 172.21.15.42:0/3598184417 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f394c097710 con 0x7f3964104930 2023-12-12T09:56:53.314 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.312+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3964068310 con 0x7f3964104930 2023-12-12T09:56:53.314 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.312+0000 7f395b7fe700 1 -- 172.21.15.42:0/3598184417 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f394c09ad30 con 0x7f3964104930 2023-12-12T09:56:53.314 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:56:53.316 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39500a1480 msgr2=0x7f39500a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.316 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39500a1480 0x7f39500a3940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3954009c60 tx=0x7f3954009260 comp rx=0 tx=0).stop 2023-12-12T09:56:53.316 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 msgr2=0x7f3964197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.316 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 0x7f3964197a30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f394c000c00 tx=0x7f394c004730 comp rx=0 tx=0).stop 2023-12-12T09:56:53.316 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 shutdown_connections 2023-12-12T09:56:53.316 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f39500a1480 0x7f39500a3940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3964102000 0x7f39641974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3964107260 0x7f396419ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 --2- 172.21.15.42:0/3598184417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3964104930 0x7f3964197a30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 >> 172.21.15.42:0/3598184417 conn(0x7f39640fb9f0 msgr2=0x7f3964103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:53.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 shutdown_connections 2023-12-12T09:56:53.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.314+0000 7f39688c9700 1 -- 172.21.15.42:0/3598184417 wait complete. 2023-12-12T09:56:53.328 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:56:53.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.455+0000 7f20e1b39700 1 -- 172.21.15.42:0/1818489269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc104990 msgr2=0x7f20dc106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.455+0000 7f20e1b39700 1 --2- 172.21.15.42:0/1818489269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc104990 0x7f20dc106d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009990 tx=0x7f20c4009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.455+0000 7f20e1b39700 1 -- 172.21.15.42:0/1818489269 shutdown_connections 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.455+0000 7f20e1b39700 1 --2- 172.21.15.42:0/1818489269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc1072c0 0x7f20dc1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.455+0000 7f20e1b39700 1 --2- 172.21.15.42:0/1818489269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20dc102060 0x7f20dc104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.455+0000 7f20e1b39700 1 --2- 172.21.15.42:0/1818489269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc104990 0x7f20dc106d80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.455+0000 7f20e1b39700 1 -- 172.21.15.42:0/1818489269 >> 172.21.15.42:0/1818489269 conn(0x7f20dc0fb9f0 msgr2=0x7f20dc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.455+0000 7f20e1b39700 1 -- 172.21.15.42:0/1818489269 shutdown_connections 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.455+0000 7f20e1b39700 1 -- 172.21.15.42:0/1818489269 wait complete. 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20e1b39700 1 Processor -- start 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20e1b39700 1 -- start start 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20e1b39700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc102060 0x7f20dc197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20e1b39700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc104990 0x7f20dc1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20e1b39700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20dc1072c0 0x7f20dc19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20e1b39700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc19efc0 con 0x7f20dc102060 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20e1b39700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc19f100 con 0x7f20dc104990 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20e1b39700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc19f290 con 0x7f20dc1072c0 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20db7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc102060 0x7f20dc197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20db7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc102060 0x7f20dc197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33746/0 (socket says 172.21.15.42:33746) 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.456+0000 7f20db7fe700 1 -- 172.21.15.42:0/3543484889 learned_addr learned my addr 172.21.15.42:0/3543484889 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20daffd700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc104990 0x7f20dc1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20dbfff700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20dc1072c0 0x7f20dc19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20db7fe700 1 -- 172.21.15.42:0/3543484889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20dc1072c0 msgr2=0x7f20dc19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20db7fe700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20dc1072c0 0x7f20dc19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20db7fe700 1 -- 172.21.15.42:0/3543484889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc104990 msgr2=0x7f20dc1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20db7fe700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc104990 0x7f20dc1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20db7fe700 1 -- 172.21.15.42:0/3543484889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40095f0 con 0x7f20dc102060 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20db7fe700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc102060 0x7f20dc197490 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00d950 tx=0x7f20cc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:53.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20d8ff9700 1 -- 172.21.15.42:0/3543484889 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc009b40 con 0x7f20dc102060 2023-12-12T09:56:53.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20d8ff9700 1 -- 172.21.15.42:0/3543484889 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20cc009ca0 con 0x7f20dc102060 2023-12-12T09:56:53.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc19f570 con 0x7f20dc102060 2023-12-12T09:56:53.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20d8ff9700 1 -- 172.21.15.42:0/3543484889 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc00f620 con 0x7f20dc102060 2023-12-12T09:56:53.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.457+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc19fa80 con 0x7f20dc102060 2023-12-12T09:56:53.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.458+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc0fd570 con 0x7f20dc102060 2023-12-12T09:56:53.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.460+0000 7f20d8ff9700 1 -- 172.21.15.42:0/3543484889 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f20cc010460 con 0x7f20dc102060 2023-12-12T09:56:53.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.460+0000 7f20d8ff9700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f20c80a1480 0x7f20c80a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.460+0000 7f20d8ff9700 1 -- 172.21.15.42:0/3543484889 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f20cc0cd2b0 con 0x7f20dc102060 2023-12-12T09:56:53.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.460+0000 7f20daffd700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f20c80a1480 0x7f20c80a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.461+0000 7f20daffd700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f20c80a1480 0x7f20c80a3940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b5c0 tx=0x7f20c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:53.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.464+0000 7f20d8ff9700 1 -- 172.21.15.42:0/3543484889 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f20cc096720 con 0x7f20dc102060 2023-12-12T09:56:53.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:53 smithi042 ceph-mon[98894]: pgmap v269: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 329 B/s wr, 2 op/s 2023-12-12T09:56:53.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:53 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/156419742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:53.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:53 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2631779004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:53.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:53 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3598184417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:56:53.671 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.669+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20dc10d010 con 0x7f20dc102060 2023-12-12T09:56:53.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.670+0000 7f20d8ff9700 1 -- 172.21.15.42:0/3543484889 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20cc0160a0 con 0x7f20dc102060 2023-12-12T09:56:53.672 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.671+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f20c80a1480 msgr2=0x7f20c80a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.671+0000 7f20e1b39700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f20c80a1480 0x7f20c80a3940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f20c400b5c0 tx=0x7f20c4005fb0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.671+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc102060 msgr2=0x7f20dc197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.671+0000 7f20e1b39700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc102060 0x7f20dc197490 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00d950 tx=0x7f20cc00dd10 comp rx=0 tx=0).stop 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.672+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 shutdown_connections 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.672+0000 7f20e1b39700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f20c80a1480 0x7f20c80a3940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.672+0000 7f20e1b39700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20dc1072c0 0x7f20dc19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.672+0000 7f20e1b39700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20dc102060 0x7f20dc197490 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.672+0000 7f20e1b39700 1 --2- 172.21.15.42:0/3543484889 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20dc104990 0x7f20dc1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.672+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 >> 172.21.15.42:0/3543484889 conn(0x7f20dc0fb9f0 msgr2=0x7f20dc1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.672+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 shutdown_connections 2023-12-12T09:56:53.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.672+0000 7f20e1b39700 1 -- 172.21.15.42:0/3543484889 wait complete. 2023-12-12T09:56:53.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:56:53.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.812+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2000697792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec104990 msgr2=0x7fd0ec106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.812+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2000697792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec104990 0x7fd0ec106d80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009990 tx=0x7fd0e0009ca0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.812+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2000697792 shutdown_connections 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.812+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2000697792 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0ec102060 0x7fd0ec104450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.812+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2000697792 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0ec1072c0 0x7fd0ec1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.812+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2000697792 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec104990 0x7fd0ec106d80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.812+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2000697792 >> 172.21.15.42:0/2000697792 conn(0x7fd0ec0fb9f0 msgr2=0x7fd0ec0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.812+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2000697792 shutdown_connections 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.812+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2000697792 wait complete. 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0f1a85700 1 Processor -- start 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0f1a85700 1 -- start start 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0f1a85700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0ec102060 0x7fd0ec197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0f1a85700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0ec104990 0x7fd0ec197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0f1a85700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec1072c0 0x7fd0ec19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0f1a85700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec19f030 con 0x7fd0ec1072c0 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0f1a85700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec19f170 con 0x7fd0ec104990 2023-12-12T09:56:53.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0f1a85700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ec19f300 con 0x7fd0ec102060 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0ebfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec1072c0 0x7fd0ec19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0ebfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec1072c0 0x7fd0ec19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33768/0 (socket says 172.21.15.42:33768) 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0ebfff700 1 -- 172.21.15.42:0/2604456113 learned_addr learned my addr 172.21.15.42:0/2604456113 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0eaffd700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0ec104990 0x7fd0ec197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.813+0000 7fd0eb7fe700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0ec102060 0x7fd0ec197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0ebfff700 1 -- 172.21.15.42:0/2604456113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0ec102060 msgr2=0x7fd0ec197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0ebfff700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0ec102060 0x7fd0ec197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0ebfff700 1 -- 172.21.15.42:0/2604456113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0ec104990 msgr2=0x7fd0ec197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0ebfff700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0ec104990 0x7fd0ec197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0ebfff700 1 -- 172.21.15.42:0/2604456113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e00095f0 con 0x7fd0ec1072c0 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0ebfff700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec1072c0 0x7fd0ec19ea60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400ef30 tx=0x7fd0e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0e8ff9700 1 -- 172.21.15.42:0/2604456113 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4018070 con 0x7fd0ec1072c0 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ec19f580 con 0x7fd0ec1072c0 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0e8ff9700 1 -- 172.21.15.42:0/2604456113 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0e400f040 con 0x7fd0ec1072c0 2023-12-12T09:56:53.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ec19fa40 con 0x7fd0ec1072c0 2023-12-12T09:56:53.819 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.814+0000 7fd0e8ff9700 1 -- 172.21.15.42:0/2604456113 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e400cd70 con 0x7fd0ec1072c0 2023-12-12T09:56:53.819 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.815+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ec0fd570 con 0x7fd0ec1072c0 2023-12-12T09:56:53.819 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.816+0000 7fd0e8ff9700 1 -- 172.21.15.42:0/2604456113 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd0e4003a00 con 0x7fd0ec1072c0 2023-12-12T09:56:53.819 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.816+0000 7fd0e8ff9700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd0d40a1480 0x7fd0d40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:53.819 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.817+0000 7fd0eb7fe700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd0d40a1480 0x7fd0d40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:53.820 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.817+0000 7fd0e8ff9700 1 -- 172.21.15.42:0/2604456113 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd0e40ccb10 con 0x7fd0ec1072c0 2023-12-12T09:56:53.820 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.817+0000 7fd0eb7fe700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd0d40a1480 0x7fd0d40a3940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009bf0 tx=0x7fd0dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:53.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:53 smithi173 ceph-mon[103308]: pgmap v269: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 329 B/s wr, 2 op/s 2023-12-12T09:56:53.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:53 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/156419742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:53.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:53 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2631779004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:53.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:53 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3598184417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:56:53.823 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:53.821+0000 7fd0e8ff9700 1 -- 172.21.15.42:0/2604456113 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd0e4095fb0 con 0x7fd0ec1072c0 2023-12-12T09:56:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:53 smithi111 ceph-mon[102565]: pgmap v269: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 329 B/s wr, 2 op/s 2023-12-12T09:56:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:53 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/156419742' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:56:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:53 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2631779004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:56:53.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:53 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3598184417' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:56:54.025 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.024+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd0ec0682e0 con 0x7fd0ec1072c0 2023-12-12T09:56:54.026 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.024+0000 7fd0e8ff9700 1 -- 172.21.15.42:0/2604456113 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd0e40995d0 con 0x7fd0ec1072c0 2023-12-12T09:56:54.026 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd0d40a1480 msgr2=0x7fd0d40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd0d40a1480 0x7fd0d40a3940 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009bf0 tx=0x7fd0dc009260 comp rx=0 tx=0).stop 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec1072c0 msgr2=0x7fd0ec19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec1072c0 0x7fd0ec19ea60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400ef30 tx=0x7fd0e400c3a0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 shutdown_connections 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd0d40a1480 0x7fd0d40a3940 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0ec1072c0 0x7fd0ec19ea60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0ec102060 0x7fd0ec197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 --2- 172.21.15.42:0/2604456113 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0ec104990 0x7fd0ec197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.026+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 >> 172.21.15.42:0/2604456113 conn(0x7fd0ec0fb9f0 msgr2=0x7fd0ec103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:54.030 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.027+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 shutdown_connections 2023-12-12T09:56:54.030 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.027+0000 7fd0f1a85700 1 -- 172.21.15.42:0/2604456113 wait complete. 2023-12-12T09:56:54.041 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T09:56:54.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 -- 172.21.15.42:0/284973835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c102010 msgr2=0x7f729c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:54.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 --2- 172.21.15.42:0/284973835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c102010 0x7f729c1088c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f729400b0d0 tx=0x7f729400b3e0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 -- 172.21.15.42:0/284973835 shutdown_connections 2023-12-12T09:56:54.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 --2- 172.21.15.42:0/284973835 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c102010 0x7f729c1088c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 --2- 172.21.15.42:0/284973835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f729c100480 0x7f729c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 --2- 172.21.15.42:0/284973835 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c101670 0x7f729c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 -- 172.21.15.42:0/284973835 >> 172.21.15.42:0/284973835 conn(0x7f729c0fb9f0 msgr2=0x7f729c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 -- 172.21.15.42:0/284973835 shutdown_connections 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 -- 172.21.15.42:0/284973835 wait complete. 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.165+0000 7f72a2922700 1 Processor -- start 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f72a2922700 1 -- start start 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f72a2922700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c100480 0x7f729c104490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f72a2922700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c101670 0x7f729c102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f72a2922700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f729c102010 0x7f729c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f72a2922700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c103800 con 0x7f729c101670 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f72a2922700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c103970 con 0x7f729c100480 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f72a2922700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f729c1a1660 con 0x7f729c102010 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f729b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c101670 0x7f729c102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:54.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f729b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c101670 0x7f729c102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33780/0 (socket says 172.21.15.42:33780) 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f729b7fe700 1 -- 172.21.15.42:0/1362154526 learned_addr learned my addr 172.21.15.42:0/1362154526 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f729bfff700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c100480 0x7f729c104490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f72a0ebf700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f729c102010 0x7f729c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f729b7fe700 1 -- 172.21.15.42:0/1362154526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f729c102010 msgr2=0x7f729c103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f729b7fe700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f729c102010 0x7f729c103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f729b7fe700 1 -- 172.21.15.42:0/1362154526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c100480 msgr2=0x7f729c104490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f729b7fe700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c100480 0x7f729c104490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.166+0000 7f729b7fe700 1 -- 172.21.15.42:0/1362154526 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7294009d00 con 0x7f729c101670 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.167+0000 7f729b7fe700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c101670 0x7f729c102b20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f728c00d950 tx=0x7f728c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.167+0000 7f72997fa700 1 -- 172.21.15.42:0/1362154526 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c00f890 con 0x7f729c101670 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.167+0000 7f72997fa700 1 -- 172.21.15.42:0/1362154526 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f728c010460 con 0x7f729c101670 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.167+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f729c1a18f0 con 0x7f729c101670 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.167+0000 7f72997fa700 1 -- 172.21.15.42:0/1362154526 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728c0195c0 con 0x7f729c101670 2023-12-12T09:56:54.171 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.167+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729c1a1e50 con 0x7f729c101670 2023-12-12T09:56:54.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.168+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f729c068310 con 0x7f729c101670 2023-12-12T09:56:54.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.169+0000 7f72997fa700 1 -- 172.21.15.42:0/1362154526 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f728c0108d0 con 0x7f729c101670 2023-12-12T09:56:54.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.169+0000 7f72997fa700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f72880a1500 0x7f72880a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:56:54.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.170+0000 7f72997fa700 1 -- 172.21.15.42:0/1362154526 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f728c0ccb70 con 0x7f729c101670 2023-12-12T09:56:54.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.170+0000 7f729bfff700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f72880a1500 0x7f72880a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:56:54.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.170+0000 7f729bfff700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f72880a1500 0x7f72880a39c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f729c1014d0 tx=0x7f7284007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:56:54.175 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.173+0000 7f72997fa700 1 -- 172.21.15.42:0/1362154526 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f728c095fe0 con 0x7f729c101670 2023-12-12T09:56:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.393+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f729c10bca0 con 0x7f729c101670 2023-12-12T09:56:54.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.393+0000 7f72997fa700 1 -- 172.21.15.42:0/1362154526 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v87) v1 ==== 81+0+591 (secure 0 0 0) 0x7f728c020080 con 0x7f729c101670 2023-12-12T09:56:54.395 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/467026676 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:54.395 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2920147377 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:54.395 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/1525180467 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:54.396 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/1433305616 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:54.396 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/1433305616 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:54.396 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3856214912 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:54.396 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/1525180467 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:54.396 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1613862950 2023-12-13T09:50:27.727252+0000 2023-12-12T09:56:54.396 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4273364865 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:54.396 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2212533654 2023-12-13T09:49:58.206187+0000 2023-12-12T09:56:54.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f72880a1500 msgr2=0x7f72880a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:54.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f72880a1500 0x7f72880a39c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f729c1014d0 tx=0x7f7284007400 comp rx=0 tx=0).stop 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c101670 msgr2=0x7f729c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c101670 0x7f729c102b20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f728c00d950 tx=0x7f728c00dc60 comp rx=0 tx=0).stop 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 shutdown_connections 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f72880a1500 0x7f72880a39c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f729c100480 0x7f729c104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f729c102010 0x7f729c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 --2- 172.21.15.42:0/1362154526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f729c101670 0x7f729c102b20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 >> 172.21.15.42:0/1362154526 conn(0x7f729c0fb9f0 msgr2=0x7f729c068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 shutdown_connections 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:56:54.395+0000 7f72a2922700 1 -- 172.21.15.42:0/1362154526 wait complete. 2023-12-12T09:56:54.398 INFO:teuthology.orchestra.run.smithi042.stderr:listed 10 entries 2023-12-12T09:56:54.409 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi042.front.sepia.ceph.com... 2023-12-12T09:56:54.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:56:54.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-12-12T09:56:54.454 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi042.stderr:ceph-fuse[153974]: fuse finished with error 0 and tester_r 0 2023-12-12T09:56:54.458 INFO:teuthology.orchestra.run:waiting for 300 2023-12-12T09:56:54.662 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:54 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3543484889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:56:54.662 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:54 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2604456113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:56:54.662 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:54 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1362154526' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-12T09:56:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:54 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3543484889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:56:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:54 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2604456113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:56:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:54 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1362154526' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-12T09:56:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:54 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3543484889' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:56:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:54 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2604456113' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:56:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:54 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1362154526' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-12T09:56:55.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:55 smithi173 ceph-mon[103308]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-12-12T09:56:55.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:55 smithi042 ceph-mon[98894]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-12-12T09:56:55.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:55 smithi111 ceph-mon[102565]: pgmap v270: 129 pgs: 129 active+clean; 579 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-12-12T09:56:57.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:57 smithi173 ceph-mon[103308]: pgmap v271: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-12T09:56:57.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:57 smithi042 ceph-mon[98894]: pgmap v271: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-12T09:56:57.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:57 smithi111 ceph-mon[102565]: pgmap v271: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-12T09:56:59.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:56:59 smithi173 ceph-mon[103308]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-12T09:56:59.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:56:59 smithi042 ceph-mon[98894]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-12T09:56:59.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:56:59 smithi111 ceph-mon[102565]: pgmap v272: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-12T09:57:00.560 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T09:57:00.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:57:00.560 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-12-12T09:57:00.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:57:00.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.776+0000 7fabd02b4700 1 -- 172.21.15.42:0/3517386894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8100380 msgr2=0x7fabc81007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:00.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.776+0000 7fabd02b4700 1 --2- 172.21.15.42:0/3517386894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8100380 0x7fabc81007a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fabc40099c0 tx=0x7fabc4009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.776+0000 7fabd02b4700 1 -- 172.21.15.42:0/3517386894 shutdown_connections 2023-12-12T09:57:00.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.776+0000 7fabd02b4700 1 --2- 172.21.15.42:0/3517386894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabc8101f10 0x7fabc81087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.776+0000 7fabd02b4700 1 --2- 172.21.15.42:0/3517386894 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8100380 0x7fabc81007a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.776+0000 7fabd02b4700 1 --2- 172.21.15.42:0/3517386894 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8101570 0x7fabc81019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.776+0000 7fabd02b4700 1 -- 172.21.15.42:0/3517386894 >> 172.21.15.42:0/3517386894 conn(0x7fabc80fb8f0 msgr2=0x7fabc80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:00.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.776+0000 7fabd02b4700 1 -- 172.21.15.42:0/3517386894 shutdown_connections 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.776+0000 7fabd02b4700 1 -- 172.21.15.42:0/3517386894 wait complete. 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.777+0000 7fabd02b4700 1 Processor -- start 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.777+0000 7fabd02b4700 1 -- start start 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.777+0000 7fabd02b4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8100380 0x7fabc8196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.777+0000 7fabd02b4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabc8101570 0x7fabc81967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.777+0000 7fabd02b4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8101f10 0x7fabc819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.777+0000 7fabd02b4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc819ddc0 con 0x7fabc8101f10 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.777+0000 7fabd02b4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc819df00 con 0x7fabc8100380 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.777+0000 7fabd02b4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc819e090 con 0x7fabc8101570 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce851700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8101f10 0x7fabc819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce050700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8100380 0x7fabc8196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce050700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8100380 0x7fabc8196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:36852/0 (socket says 172.21.15.42:36852) 2023-12-12T09:57:00.782 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce050700 1 -- 172.21.15.42:0/554465935 learned_addr learned my addr 172.21.15.42:0/554465935 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce851700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8101f10 0x7fabc819d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33796/0 (socket says 172.21.15.42:33796) 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabcd84f700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabc8101570 0x7fabc81967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce851700 1 -- 172.21.15.42:0/554465935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabc8101570 msgr2=0x7fabc81967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce851700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabc8101570 0x7fabc81967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce851700 1 -- 172.21.15.42:0/554465935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8100380 msgr2=0x7fabc8196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce851700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8100380 0x7fabc8196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce851700 1 -- 172.21.15.42:0/554465935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40095f0 con 0x7fabc8101f10 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce050700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8100380 0x7fabc8196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabce851700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8101f10 0x7fabc819d7f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fabc000cc50 tx=0x7fabc000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabbb7fe700 1 -- 172.21.15.42:0/554465935 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc000aa90 con 0x7fabc8101f10 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc8068670 con 0x7fabc8101f10 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc8068b80 con 0x7fabc8101f10 2023-12-12T09:57:00.783 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabbb7fe700 1 -- 172.21.15.42:0/554465935 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fabc0007a80 con 0x7fabc8101f10 2023-12-12T09:57:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.778+0000 7fabbb7fe700 1 -- 172.21.15.42:0/554465935 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc001b660 con 0x7fabc8101f10 2023-12-12T09:57:00.784 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.779+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac005320 con 0x7fabc8101f10 2023-12-12T09:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.780+0000 7fabbb7fe700 1 -- 172.21.15.42:0/554465935 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fabc000abf0 con 0x7fabc8101f10 2023-12-12T09:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.781+0000 7fabbb7fe700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fabb40a1480 0x7fabb40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.781+0000 7fabce050700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fabb40a1480 0x7fabb40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.781+0000 7fabbb7fe700 1 -- 172.21.15.42:0/554465935 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fabc00ccbf0 con 0x7fabc8101f10 2023-12-12T09:57:00.785 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.782+0000 7fabce050700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fabb40a1480 0x7fabb40a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fabc4005b40 tx=0x7fabc4005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:00.788 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.786+0000 7fabbb7fe700 1 -- 172.21.15.42:0/554465935 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fabc0096060 con 0x7fabc8101f10 2023-12-12T09:57:00.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.990+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabac0055d0 con 0x7fabc8101f10 2023-12-12T09:57:00.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.991+0000 7fabbb7fe700 1 -- 172.21.15.42:0/554465935 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7fabc0099680 con 0x7fabc8101f10 2023-12-12T09:57:00.994 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:00.994 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:57:00.995 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fabb40a1480 msgr2=0x7fabb40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fabb40a1480 0x7fabb40a3940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fabc4005b40 tx=0x7fabc4005ad0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8101f10 msgr2=0x7fabc819d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8101f10 0x7fabc819d7f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fabc000cc50 tx=0x7fabc000cf60 comp rx=0 tx=0).stop 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 shutdown_connections 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fabb40a1480 0x7fabb40a3940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fabc8100380 0x7fabc8196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabc8101f10 0x7fabc819d7f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 --2- 172.21.15.42:0/554465935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabc8101570 0x7fabc81967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 >> 172.21.15.42:0/554465935 conn(0x7fabc80fb8f0 msgr2=0x7fabc81048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 shutdown_connections 2023-12-12T09:57:00.996 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:00.993+0000 7fabd02b4700 1 -- 172.21.15.42:0/554465935 wait complete. 2023-12-12T09:57:01.008 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.131+0000 7f97ed49c700 1 -- 172.21.15.42:0/373093997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 msgr2=0x7f97e81087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.131+0000 7f97ed49c700 1 --2- 172.21.15.42:0/373093997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 0x7f97e81087c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f97e000b0d0 tx=0x7f97e000b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.131+0000 7f97ed49c700 1 -- 172.21.15.42:0/373093997 shutdown_connections 2023-12-12T09:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.131+0000 7f97ed49c700 1 --2- 172.21.15.42:0/373093997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 0x7f97e81087c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.131+0000 7f97ed49c700 1 --2- 172.21.15.42:0/373093997 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97e8100380 0x7f97e81007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.131+0000 7f97ed49c700 1 --2- 172.21.15.42:0/373093997 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97e8101570 0x7f97e81019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.131+0000 7f97ed49c700 1 -- 172.21.15.42:0/373093997 >> 172.21.15.42:0/373093997 conn(0x7f97e80fb8f0 msgr2=0x7f97e80fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:01.136 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.131+0000 7f97ed49c700 1 -- 172.21.15.42:0/373093997 shutdown_connections 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.131+0000 7f97ed49c700 1 -- 172.21.15.42:0/373093997 wait complete. 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97ed49c700 1 Processor -- start 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97ed49c700 1 -- start start 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97ed49c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97e8100380 0x7f97e810ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97ed49c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97e8101570 0x7f97e81092d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97ed49c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 0x7f97e8109810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97ed49c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8109fb0 con 0x7f97e8101f10 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97ed49c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e810a120 con 0x7f97e8100380 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97ed49c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e81a1540 con 0x7f97e8101570 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97e77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 0x7f97e8109810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97e77fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 0x7f97e8109810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33810/0 (socket says 172.21.15.42:33810) 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97e77fe700 1 -- 172.21.15.42:0/2124609882 learned_addr learned my addr 172.21.15.42:0/2124609882 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97e6ffd700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97e8100380 0x7f97e810ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.132+0000 7f97e67fc700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97e8101570 0x7f97e81092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:01.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97e77fe700 1 -- 172.21.15.42:0/2124609882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97e8101570 msgr2=0x7f97e81092d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97e77fe700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97e8101570 0x7f97e81092d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97e77fe700 1 -- 172.21.15.42:0/2124609882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97e8100380 msgr2=0x7f97e810ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97e77fe700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97e8100380 0x7f97e810ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97e77fe700 1 -- 172.21.15.42:0/2124609882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e0009d00 con 0x7f97e8101f10 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97e77fe700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 0x7f97e8109810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97e0007b60 tx=0x7f97e0008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97cffff700 1 -- 172.21.15.42:0/2124609882 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e000e040 con 0x7f97e8101f10 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97cffff700 1 -- 172.21.15.42:0/2124609882 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e001d930 con 0x7f97e8101f10 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e81a1770 con 0x7f97e8101f10 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97cffff700 1 -- 172.21.15.42:0/2124609882 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e001ba00 con 0x7f97e8101f10 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.133+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e81a1b40 con 0x7f97e8101f10 2023-12-12T09:57:01.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.134+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e81077b0 con 0x7f97e8101f10 2023-12-12T09:57:01.139 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.135+0000 7f97cffff700 1 -- 172.21.15.42:0/2124609882 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f97e001daa0 con 0x7f97e8101f10 2023-12-12T09:57:01.139 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.136+0000 7f97cffff700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97d00a1480 0x7f97d00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:01.139 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.136+0000 7f97e6ffd700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97d00a1480 0x7f97d00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:01.139 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.136+0000 7f97cffff700 1 -- 172.21.15.42:0/2124609882 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f97e00ce8d0 con 0x7f97e8101f10 2023-12-12T09:57:01.139 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.136+0000 7f97e6ffd700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97d00a1480 0x7f97d00a3940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97e81013d0 tx=0x7f97d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:01.141 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.139+0000 7f97cffff700 1 -- 172.21.15.42:0/2124609882 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f97e0097d70 con 0x7f97e8101f10 2023-12-12T09:57:01.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.344+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97e810ce50 con 0x7f97e8101f10 2023-12-12T09:57:01.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.346+0000 7f97cffff700 1 -- 172.21.15.42:0/2124609882 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7f97e009b390 con 0x7f97e8101f10 2023-12-12T09:57:01.349 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:01.349 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:57:01.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97d00a1480 msgr2=0x7f97d00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:01.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97d00a1480 0x7f97d00a3940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f97e81013d0 tx=0x7f97d8009260 comp rx=0 tx=0).stop 2023-12-12T09:57:01.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 msgr2=0x7f97e8109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:01.352 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 0x7f97e8109810 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f97e0007b60 tx=0x7f97e0008ec0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.352 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 shutdown_connections 2023-12-12T09:57:01.352 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f97d00a1480 0x7f97d00a3940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.352 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97e8100380 0x7f97e810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.352 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97e8101f10 0x7f97e8109810 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.352 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 --2- 172.21.15.42:0/2124609882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97e8101570 0x7f97e81092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.352 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 >> 172.21.15.42:0/2124609882 conn(0x7f97e80fb8f0 msgr2=0x7f97e81048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:01.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 shutdown_connections 2023-12-12T09:57:01.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.348+0000 7f97ed49c700 1 -- 172.21.15.42:0/2124609882 wait complete. 2023-12-12T09:57:01.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.493+0000 7fd05a23f700 1 -- 172.21.15.42:0/1533728514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 msgr2=0x7fd0541088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.493+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1533728514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 0x7fd0541088d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00b0d0 tx=0x7fd04c00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.493+0000 7fd05a23f700 1 -- 172.21.15.42:0/1533728514 shutdown_connections 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.493+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1533728514 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 0x7fd0541088d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.493+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1533728514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd054100490 0x7fd0541008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.493+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1533728514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd054101680 0x7fd054101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.493+0000 7fd05a23f700 1 -- 172.21.15.42:0/1533728514 >> 172.21.15.42:0/1533728514 conn(0x7fd0540fba20 msgr2=0x7fd0540fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.493+0000 7fd05a23f700 1 -- 172.21.15.42:0/1533728514 shutdown_connections 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.493+0000 7fd05a23f700 1 -- 172.21.15.42:0/1533728514 wait complete. 2023-12-12T09:57:01.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd05a23f700 1 Processor -- start 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd05a23f700 1 -- start start 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd05a23f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd054100490 0x7fd05406a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd05a23f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd054101680 0x7fd05419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd05a23f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 0x7fd0541a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd05a23f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0541a12e0 con 0x7fd054102020 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd05a23f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0541a1450 con 0x7fd054100490 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd05a23f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0541a1610 con 0x7fd054101680 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd053fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 0x7fd0541a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd053fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 0x7fd0541a0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33824/0 (socket says 172.21.15.42:33824) 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd053fff700 1 -- 172.21.15.42:0/1388562285 learned_addr learned my addr 172.21.15.42:0/1388562285 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.494+0000 7fd052ffd700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd054101680 0x7fd05419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd0537fe700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd054100490 0x7fd05406a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd053fff700 1 -- 172.21.15.42:0/1388562285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd054101680 msgr2=0x7fd05419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:01.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd053fff700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd054101680 0x7fd05419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd053fff700 1 -- 172.21.15.42:0/1388562285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd054100490 msgr2=0x7fd05406a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd053fff700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd054100490 0x7fd05406a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd053fff700 1 -- 172.21.15.42:0/1388562285 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c009d00 con 0x7fd054102020 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd0537fe700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd054100490 0x7fd05406a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd053fff700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 0x7fd0541a0c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd04c007b60 tx=0x7fd04c008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd050ff9700 1 -- 172.21.15.42:0/1388562285 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00e040 con 0x7fd054102020 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0541a1890 con 0x7fd054102020 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd050ff9700 1 -- 172.21.15.42:0/1388562285 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd04c00be30 con 0x7fd054102020 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd050ff9700 1 -- 172.21.15.42:0/1388562285 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04c01bae0 con 0x7fd054102020 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.495+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0541a1cc0 con 0x7fd054102020 2023-12-12T09:57:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.496+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd054107840 con 0x7fd054102020 2023-12-12T09:57:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.498+0000 7fd050ff9700 1 -- 172.21.15.42:0/1388562285 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd04c01d670 con 0x7fd054102020 2023-12-12T09:57:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.498+0000 7fd050ff9700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd03c0a1480 0x7fd03c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.498+0000 7fd050ff9700 1 -- 172.21.15.42:0/1388562285 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd04c0cef20 con 0x7fd054102020 2023-12-12T09:57:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.498+0000 7fd0537fe700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd03c0a1480 0x7fd03c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:01.501 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.499+0000 7fd0537fe700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd03c0a1480 0x7fd03c0a3940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd0541014e0 tx=0x7fd048009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:01.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.501+0000 7fd050ff9700 1 -- 172.21.15.42:0/1388562285 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd04c098420 con 0x7fd054102020 2023-12-12T09:57:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:01 smithi042 ceph-mon[98894]: pgmap v273: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-12T09:57:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:01 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/554465935' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:01.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:01 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2124609882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:01.751 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:01.749+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd054068760 con 0x7fd054102020 2023-12-12T09:57:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:01 smithi173 ceph-mon[103308]: pgmap v273: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-12T09:57:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:01 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/554465935' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:01 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2124609882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:01.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:01 smithi111 ceph-mon[102565]: pgmap v273: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-12T09:57:01.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:01 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/554465935' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:01.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:01 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2124609882' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:02.546 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.544+0000 7fd050ff9700 1 -- 172.21.15.42:0/1388562285 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd04c09ba40 con 0x7fd054102020 2023-12-12T09:57:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd03c0a1480 msgr2=0x7fd03c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd03c0a1480 0x7fd03c0a3940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd0541014e0 tx=0x7fd048009260 comp rx=0 tx=0).stop 2023-12-12T09:57:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 msgr2=0x7fd0541a0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 0x7fd0541a0c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd04c007b60 tx=0x7fd04c008f10 comp rx=0 tx=0).stop 2023-12-12T09:57:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 shutdown_connections 2023-12-12T09:57:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd03c0a1480 0x7fd03c0a3940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd054100490 0x7fd05406a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd054102020 0x7fd0541a0c80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 --2- 172.21.15.42:0/1388562285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd054101680 0x7fd05419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 >> 172.21.15.42:0/1388562285 conn(0x7fd0540fba20 msgr2=0x7fd054104980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:02.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 shutdown_connections 2023-12-12T09:57:02.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.547+0000 7fd05a23f700 1 -- 172.21.15.42:0/1388562285 wait complete. 2023-12-12T09:57:02.564 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2023-12-12T09:57:02.565 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2023-12-12T09:57:02.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.699+0000 7fbb83ea4700 1 -- 172.21.15.42:0/594231666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c102790 msgr2=0x7fbb7c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:02.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.699+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/594231666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c102790 0x7fbb7c102bf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb780099c0 tx=0x7fbb78009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.699+0000 7fbb83ea4700 1 -- 172.21.15.42:0/594231666 shutdown_connections 2023-12-12T09:57:02.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.699+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/594231666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb7c103130 0x7fbb7c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.699+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/594231666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c1015a0 0x7fbb7c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.703 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.699+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/594231666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c102790 0x7fbb7c102bf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.699+0000 7fbb83ea4700 1 -- 172.21.15.42:0/594231666 >> 172.21.15.42:0/594231666 conn(0x7fbb7c0fcb10 msgr2=0x7fbb7c0fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.699+0000 7fbb83ea4700 1 -- 172.21.15.42:0/594231666 shutdown_connections 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.699+0000 7fbb83ea4700 1 -- 172.21.15.42:0/594231666 wait complete. 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb83ea4700 1 Processor -- start 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb83ea4700 1 -- start start 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb83ea4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb7c1015a0 0x7fbb7c1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb83ea4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c102790 0x7fbb7c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb83ea4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c103130 0x7fbb7c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb83ea4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c19f010 con 0x7fbb7c103130 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb83ea4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c19f150 con 0x7fbb7c1015a0 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb83ea4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c19f2e0 con 0x7fbb7c102790 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb82441700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c103130 0x7fbb7c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:02.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb82441700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c103130 0x7fbb7c19ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33838/0 (socket says 172.21.15.42:33838) 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb82441700 1 -- 172.21.15.42:0/1029394438 learned_addr learned my addr 172.21.15.42:0/1029394438 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb81c40700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb7c1015a0 0x7fbb7c1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb8143f700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c102790 0x7fbb7c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb82441700 1 -- 172.21.15.42:0/1029394438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c102790 msgr2=0x7fbb7c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb82441700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c102790 0x7fbb7c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb82441700 1 -- 172.21.15.42:0/1029394438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb7c1015a0 msgr2=0x7fbb7c1974e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb82441700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb7c1015a0 0x7fbb7c1974e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb82441700 1 -- 172.21.15.42:0/1029394438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb780095f0 con 0x7fbb7c103130 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.700+0000 7fbb82441700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c103130 0x7fbb7c19ea40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbb74007ee0 tx=0x7fbb7400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.701+0000 7fbb72ffd700 1 -- 172.21.15.42:0/1029394438 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74017070 con 0x7fbb7c103130 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.701+0000 7fbb72ffd700 1 -- 172.21.15.42:0/1029394438 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7400f040 con 0x7fbb7c103130 2023-12-12T09:57:02.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.701+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c19f5c0 con 0x7fbb7c103130 2023-12-12T09:57:02.706 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.701+0000 7fbb72ffd700 1 -- 172.21.15.42:0/1029394438 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74005580 con 0x7fbb7c103130 2023-12-12T09:57:02.706 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.701+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c19fa80 con 0x7fbb7c103130 2023-12-12T09:57:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.702+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c108a70 con 0x7fbb7c103130 2023-12-12T09:57:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.703+0000 7fbb72ffd700 1 -- 172.21.15.42:0/1029394438 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fbb7400db20 con 0x7fbb7c103130 2023-12-12T09:57:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.703+0000 7fbb72ffd700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb680a1480 0x7fbb680a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.703+0000 7fbb72ffd700 1 -- 172.21.15.42:0/1029394438 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbb740cc650 con 0x7fbb7c103130 2023-12-12T09:57:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.703+0000 7fbb81c40700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb680a1480 0x7fbb680a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.704+0000 7fbb81c40700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb680a1480 0x7fbb680a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c1025f0 tx=0x7fbb6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:02.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.708+0000 7fbb72ffd700 1 -- 172.21.15.42:0/1029394438 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbb74095ac0 con 0x7fbb7c103130 2023-12-12T09:57:02.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:02 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1388562285' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-12T09:57:02.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:02 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1388562285' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-12T09:57:02.959 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:02.957+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7fbb7c108450 con 0x7fbb7c103130 2023-12-12T09:57:02.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:02 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1388562285' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-12T09:57:03.266 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.264+0000 7fbb72ffd700 1 -- 172.21.15.42:0/1029394438 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7fbb740990e0 con 0x7fbb7c103130 2023-12-12T09:57:03.269 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb680a1480 msgr2=0x7fbb680a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:03.269 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb680a1480 0x7fbb680a3940 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c1025f0 tx=0x7fbb6c00b410 comp rx=0 tx=0).stop 2023-12-12T09:57:03.269 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c103130 msgr2=0x7fbb7c19ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:03.269 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c103130 0x7fbb7c19ea40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbb74007ee0 tx=0x7fbb7400d3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 shutdown_connections 2023-12-12T09:57:03.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb680a1480 0x7fbb680a3940 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb7c1015a0 0x7fbb7c1974e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb7c103130 0x7fbb7c19ea40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 --2- 172.21.15.42:0/1029394438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb7c102790 0x7fbb7c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.266+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 >> 172.21.15.42:0/1029394438 conn(0x7fbb7c0fcb10 msgr2=0x7fbb7c105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:03.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.267+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 shutdown_connections 2023-12-12T09:57:03.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.267+0000 7fbb83ea4700 1 -- 172.21.15.42:0/1029394438 wait complete. 2023-12-12T09:57:03.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-12-12T09:57:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.413+0000 7f84782ed700 1 -- 172.21.15.42:0/227310085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8470101080 msgr2=0x7f8470108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.413+0000 7f84782ed700 1 --2- 172.21.15.42:0/227310085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8470101080 0x7f8470108920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f846800b0a0 tx=0x7f846800b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 -- 172.21.15.42:0/227310085 shutdown_connections 2023-12-12T09:57:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 --2- 172.21.15.42:0/227310085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8470101080 0x7f8470108920 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 --2- 172.21.15.42:0/227310085 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8470101b00 0x7f8470101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 --2- 172.21.15.42:0/227310085 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84701006c0 0x7f8470100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 -- 172.21.15.42:0/227310085 >> 172.21.15.42:0/227310085 conn(0x7f84700fb170 msgr2=0x7f84700fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 -- 172.21.15.42:0/227310085 shutdown_connections 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 -- 172.21.15.42:0/227310085 wait complete. 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 Processor -- start 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 -- start start 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84701006c0 0x7f84701964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8470101080 0x7f8470196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.414+0000 7f84782ed700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8470101b00 0x7f847019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f84782ed700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847019e000 con 0x7f84701006c0 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f84782ed700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847019e140 con 0x7f8470101080 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f84782ed700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847019e2d0 con 0x7f8470101b00 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8476089700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84701006c0 0x7f84701964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8476089700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84701006c0 0x7f84701964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33848/0 (socket says 172.21.15.42:33848) 2023-12-12T09:57:03.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8476089700 1 -- 172.21.15.42:0/984733294 learned_addr learned my addr 172.21.15.42:0/984733294 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f847688a700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8470101b00 0x7f847019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8475888700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8470101080 0x7f8470196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8476089700 1 -- 172.21.15.42:0/984733294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8470101b00 msgr2=0x7f847019da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8476089700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8470101b00 0x7f847019da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8476089700 1 -- 172.21.15.42:0/984733294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8470101080 msgr2=0x7f8470196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8476089700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8470101080 0x7f8470196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8476089700 1 -- 172.21.15.42:0/984733294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468009d00 con 0x7f84701006c0 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f8476089700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84701006c0 0x7f84701964d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f846400d980 tx=0x7f846400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f84637fe700 1 -- 172.21.15.42:0/984733294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464004220 con 0x7f84701006c0 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f84637fe700 1 -- 172.21.15.42:0/984733294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8464007640 con 0x7f84701006c0 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847019e5b0 con 0x7f84701006c0 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f84637fe700 1 -- 172.21.15.42:0/984733294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8464005690 con 0x7f84701006c0 2023-12-12T09:57:03.422 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.415+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847019eb10 con 0x7f84701006c0 2023-12-12T09:57:03.423 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.416+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84701078e0 con 0x7f84701006c0 2023-12-12T09:57:03.423 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.418+0000 7f84637fe700 1 -- 172.21.15.42:0/984733294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8464004750 con 0x7f84701006c0 2023-12-12T09:57:03.423 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.419+0000 7f84637fe700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f845c0a1500 0x7f845c0a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:03.424 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.419+0000 7f8475888700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f845c0a1500 0x7f845c0a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:03.424 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.419+0000 7f84637fe700 1 -- 172.21.15.42:0/984733294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f84640ccf10 con 0x7f84701006c0 2023-12-12T09:57:03.424 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.419+0000 7f8475888700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f845c0a1500 0x7f845c0a39c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f846c005950 tx=0x7f846c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:03.424 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.422+0000 7f84637fe700 1 -- 172.21.15.42:0/984733294 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8464096320 con 0x7f84701006c0 2023-12-12T09:57:03.675 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:03.673+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f847010bf40 con 0x7f84701006c0 2023-12-12T09:57:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:03 smithi173 ceph-mon[103308]: pgmap v274: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-12T09:57:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:03 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1388562285' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-12T09:57:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:03 smithi173 ceph-mon[103308]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:57:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:03 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1029394438' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-12T09:57:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:03 smithi173 ceph-mon[103308]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-12T09:57:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:03 smithi173 ceph-mon[103308]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-12T09:57:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:03 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1029394438' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-12T09:57:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:03 smithi173 ceph-mon[103308]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-12T09:57:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:03 smithi173 ceph-mon[103308]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-12T09:57:03.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:03 smithi042 ceph-mon[98894]: pgmap v274: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-12T09:57:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:03 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1388562285' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-12T09:57:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:03 smithi042 ceph-mon[98894]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:57:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:03 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1029394438' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-12T09:57:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:03 smithi042 ceph-mon[98894]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-12T09:57:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:03 smithi042 ceph-mon[98894]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-12T09:57:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:03 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1029394438' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-12T09:57:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:03 smithi042 ceph-mon[98894]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-12T09:57:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:03 smithi042 ceph-mon[98894]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-12T09:57:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:03 smithi111 ceph-mon[102565]: pgmap v274: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-12T09:57:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:03 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1388562285' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-12T09:57:03.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:03 smithi111 ceph-mon[102565]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-12T09:57:03.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:03 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1029394438' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2023-12-12T09:57:03.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:03 smithi111 ceph-mon[102565]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-12T09:57:03.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:03 smithi111 ceph-mon[102565]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-12T09:57:03.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:03 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1029394438' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2023-12-12T09:57:03.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:03 smithi111 ceph-mon[102565]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-12T09:57:03.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:03 smithi111 ceph-mon[102565]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-12T09:57:04.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.271+0000 7f84637fe700 1 -- 172.21.15.42:0/984733294 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f84640160c0 con 0x7f84701006c0 2023-12-12T09:57:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f845c0a1500 msgr2=0x7f845c0a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f845c0a1500 0x7f845c0a39c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f846c005950 tx=0x7f846c00b410 comp rx=0 tx=0).stop 2023-12-12T09:57:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84701006c0 msgr2=0x7f84701964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84701006c0 0x7f84701964d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f846400d980 tx=0x7f846400dc90 comp rx=0 tx=0).stop 2023-12-12T09:57:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 shutdown_connections 2023-12-12T09:57:04.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f845c0a1500 0x7f845c0a39c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8470101b00 0x7f847019da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84701006c0 0x7f84701964d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 --2- 172.21.15.42:0/984733294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8470101080 0x7f8470196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 >> 172.21.15.42:0/984733294 conn(0x7f84700fb170 msgr2=0x7f8470104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 shutdown_connections 2023-12-12T09:57:04.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.274+0000 7f84782ed700 1 -- 172.21.15.42:0/984733294 wait complete. 2023-12-12T09:57:04.293 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-12-12T09:57:04.430 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1471347736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18101680 msgr2=0x7f8d18101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:04.430 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1471347736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18101680 0x7f8d18101ae0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8d080099c0 tx=0x7f8d08009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1471347736 shutdown_connections 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1471347736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d18102020 0x7f8d181088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1471347736 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d18100490 0x7f8d181008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1471347736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18101680 0x7f8d18101ae0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1471347736 >> 172.21.15.42:0/1471347736 conn(0x7f8d180fba20 msgr2=0x7f8d180fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1471347736 shutdown_connections 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1471347736 wait complete. 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.423+0000 7f8d1d3e5700 1 Processor -- start 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d1d3e5700 1 -- start start 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d1d3e5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18100490 0x7f8d181963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d1d3e5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d18101680 0x7f8d18196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d1d3e5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d18102020 0x7f8d1819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d1d3e5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1819def0 con 0x7f8d18100490 2023-12-12T09:57:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d1d3e5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1819e030 con 0x7f8d18101680 2023-12-12T09:57:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d1d3e5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1819e1c0 con 0x7f8d18102020 2023-12-12T09:57:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d16ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18100490 0x7f8d181963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d16ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18100490 0x7f8d181963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33862/0 (socket says 172.21.15.42:33862) 2023-12-12T09:57:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d16ffd700 1 -- 172.21.15.42:0/1834803000 learned_addr learned my addr 172.21.15.42:0/1834803000 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d177fe700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d18102020 0x7f8d1819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d167fc700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d18101680 0x7f8d18196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d16ffd700 1 -- 172.21.15.42:0/1834803000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d18102020 msgr2=0x7f8d1819d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d16ffd700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d18102020 0x7f8d1819d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d16ffd700 1 -- 172.21.15.42:0/1834803000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d18101680 msgr2=0x7f8d18196900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d16ffd700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d18101680 0x7f8d18196900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d16ffd700 1 -- 172.21.15.42:0/1834803000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d080095f0 con 0x7f8d18100490 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d177fe700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d18102020 0x7f8d1819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d16ffd700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18100490 0x7f8d181963c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000d980 tx=0x7f8d0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.424+0000 7f8d0ffff700 1 -- 172.21.15.42:0/1834803000 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d00004020 con 0x7f8d18100490 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.425+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1819e4a0 con 0x7f8d18100490 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.425+0000 7f8d0ffff700 1 -- 172.21.15.42:0/1834803000 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d00009730 con 0x7f8d18100490 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.425+0000 7f8d0ffff700 1 -- 172.21.15.42:0/1834803000 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d00003a30 con 0x7f8d18100490 2023-12-12T09:57:04.433 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.425+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1819ea00 con 0x7f8d18100490 2023-12-12T09:57:04.434 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.425+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d18107960 con 0x7f8d18100490 2023-12-12T09:57:04.434 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.427+0000 7f8d0ffff700 1 -- 172.21.15.42:0/1834803000 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f8d000098a0 con 0x7f8d18100490 2023-12-12T09:57:04.434 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.427+0000 7f8d0ffff700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d040a1480 0x7f8d040a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:04.434 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.427+0000 7f8d167fc700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d040a1480 0x7f8d040a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:04.434 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.427+0000 7f8d0ffff700 1 -- 172.21.15.42:0/1834803000 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8d000ccf40 con 0x7f8d18100490 2023-12-12T09:57:04.434 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.428+0000 7f8d167fc700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d040a1480 0x7f8d040a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009990 tx=0x7f8d08005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:04.435 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.431+0000 7f8d0ffff700 1 -- 172.21.15.42:0/1834803000 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8d000963b0 con 0x7f8d18100490 2023-12-12T09:57:04.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:04.681+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f8d18068760 con 0x7f8d18100490 2023-12-12T09:57:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:04 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/984733294' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-12T09:57:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:04 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/984733294' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-12T09:57:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:04 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] up:active 2023-12-12T09:57:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:04 smithi173 ceph-mon[103308]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-12T09:57:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:04 smithi173 ceph-mon[103308]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-12T09:57:04.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:04 smithi173 ceph-mon[103308]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-12-12T09:57:04.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:04 smithi173 ceph-mon[103308]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-12T09:57:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:04 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/984733294' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-12T09:57:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:04 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/984733294' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-12T09:57:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:04 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] up:active 2023-12-12T09:57:04.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:04 smithi042 ceph-mon[98894]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-12T09:57:04.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:04 smithi042 ceph-mon[98894]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-12T09:57:04.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:04 smithi042 ceph-mon[98894]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-12-12T09:57:04.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:04 smithi042 ceph-mon[98894]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-12T09:57:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:04 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/984733294' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-12T09:57:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:04 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/984733294' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-12T09:57:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:04 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] up:active 2023-12-12T09:57:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:04 smithi111 ceph-mon[102565]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-12T09:57:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:04 smithi111 ceph-mon[102565]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2023-12-12T09:57:04.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:04 smithi111 ceph-mon[102565]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:creating} 9 up:standby 2023-12-12T09:57:04.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:04 smithi111 ceph-mon[102565]: daemon mds.c is now active in filesystem cephfs as rank 2 2023-12-12T09:57:05.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.278+0000 7f8d0ffff700 1 -- 172.21.15.42:0/1834803000 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7f8d000999d0 con 0x7f8d18100490 2023-12-12T09:57:05.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d040a1480 msgr2=0x7f8d040a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d040a1480 0x7f8d040a3940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009990 tx=0x7f8d08005ad0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18100490 msgr2=0x7f8d181963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18100490 0x7f8d181963c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8d0000d980 tx=0x7f8d0000dc90 comp rx=0 tx=0).stop 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 shutdown_connections 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d040a1480 0x7f8d040a3940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d18102020 0x7f8d1819d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d18100490 0x7f8d181963c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 --2- 172.21.15.42:0/1834803000 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d18101680 0x7f8d18196900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 >> 172.21.15.42:0/1834803000 conn(0x7f8d180fba20 msgr2=0x7f8d18104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 shutdown_connections 2023-12-12T09:57:05.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.281+0000 7f8d1d3e5700 1 -- 172.21.15.42:0/1834803000 wait complete. 2023-12-12T09:57:05.302 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2023-12-12T09:57:05.302 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-12-12T09:57:05.440 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.436+0000 7fd50fb44700 1 -- 172.21.15.42:0/752224362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 msgr2=0x7fd5081008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.440 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.436+0000 7fd50fb44700 1 --2- 172.21.15.42:0/752224362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 0x7fd5081008a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd5040099c0 tx=0x7fd504009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.440 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.436+0000 7fd50fb44700 1 -- 172.21.15.42:0/752224362 shutdown_connections 2023-12-12T09:57:05.440 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.436+0000 7fd50fb44700 1 --2- 172.21.15.42:0/752224362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508102010 0x7fd5081088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.440 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.436+0000 7fd50fb44700 1 --2- 172.21.15.42:0/752224362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 0x7fd5081008a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.440 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.436+0000 7fd50fb44700 1 --2- 172.21.15.42:0/752224362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd508101670 0x7fd508101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.436+0000 7fd50fb44700 1 -- 172.21.15.42:0/752224362 >> 172.21.15.42:0/752224362 conn(0x7fd5080fb9f0 msgr2=0x7fd5080fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.436+0000 7fd50fb44700 1 -- 172.21.15.42:0/752224362 shutdown_connections 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.436+0000 7fd50fb44700 1 -- 172.21.15.42:0/752224362 wait complete. 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50fb44700 1 Processor -- start 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50fb44700 1 -- start start 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50fb44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 0x7fd508196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50fb44700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd508101670 0x7fd508196a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50fb44700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508102010 0x7fd50819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50fb44700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50819e080 con 0x7fd508100480 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50fb44700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50819e1c0 con 0x7fd508102010 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50fb44700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50819e350 con 0x7fd508101670 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d8e0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 0x7fd508196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d8e0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 0x7fd508196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33876/0 (socket says 172.21.15.42:33876) 2023-12-12T09:57:05.441 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d8e0700 1 -- 172.21.15.42:0/129424762 learned_addr learned my addr 172.21.15.42:0/129424762 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50e0e1700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508102010 0x7fd50819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d0df700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd508101670 0x7fd508196a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d8e0700 1 -- 172.21.15.42:0/129424762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd508101670 msgr2=0x7fd508196a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d8e0700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd508101670 0x7fd508196a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d8e0700 1 -- 172.21.15.42:0/129424762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508102010 msgr2=0x7fd50819dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d8e0700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508102010 0x7fd50819dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d8e0700 1 -- 172.21.15.42:0/129424762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5040095f0 con 0x7fd508100480 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50e0e1700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508102010 0x7fd50819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.437+0000 7fd50d8e0700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 0x7fd508196550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd504009f50 tx=0x7fd504003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.438+0000 7fd4faffd700 1 -- 172.21.15.42:0/129424762 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50401d070 con 0x7fd508100480 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.438+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50819e5d0 con 0x7fd508100480 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.438+0000 7fd4faffd700 1 -- 172.21.15.42:0/129424762 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd504022470 con 0x7fd508100480 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.438+0000 7fd4faffd700 1 -- 172.21.15.42:0/129424762 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400f6c0 con 0x7fd508100480 2023-12-12T09:57:05.442 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.438+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50819ea00 con 0x7fd508100480 2023-12-12T09:57:05.443 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.438+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd508068310 con 0x7fd508100480 2023-12-12T09:57:05.443 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.440+0000 7fd4faffd700 1 -- 172.21.15.42:0/129424762 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7fd5040225e0 con 0x7fd508100480 2023-12-12T09:57:05.443 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.440+0000 7fd4faffd700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4f40a1480 0x7fd4f40a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:05.444 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.440+0000 7fd4faffd700 1 -- 172.21.15.42:0/129424762 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd5040ce970 con 0x7fd508100480 2023-12-12T09:57:05.444 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.440+0000 7fd50d0df700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4f40a1480 0x7fd4f40a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:05.444 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.441+0000 7fd50d0df700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4f40a1480 0x7fd4f40a3940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc006fd0 tx=0x7fd4fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:05.446 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.444+0000 7fd4faffd700 1 -- 172.21.15.42:0/129424762 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd504097e70 con 0x7fd508100480 2023-12-12T09:57:05.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-mon[98894]: pgmap v275: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-12T09:57:05.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1834803000' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-12T09:57:05.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1834803000' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-12T09:57:05.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] up:active 2023-12-12T09:57:05.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-mon[98894]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-12T09:57:05.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-mon[98894]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-12-12T09:57:05.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-mon[98894]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2023-12-12T09:57:05.560 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-mon[98894]: daemon mds.h is now active in filesystem cephfs as rank 3 2023-12-12T09:57:05.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.680+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fd508107470 con 0x7fd4f40a1480 2023-12-12T09:57:05.730 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.728+0000 7fd4faffd700 1 -- 172.21.15.42:0/129424762 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd508107470 con 0x7fd4f40a1480 2023-12-12T09:57:05.734 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4f40a1480 msgr2=0x7fd4f40a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.734 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4f40a1480 0x7fd4f40a3940 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc006fd0 tx=0x7fd4fc008040 comp rx=0 tx=0).stop 2023-12-12T09:57:05.734 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 msgr2=0x7fd508196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.735 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 0x7fd508196550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd504009f50 tx=0x7fd504003970 comp rx=0 tx=0).stop 2023-12-12T09:57:05.735 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 shutdown_connections 2023-12-12T09:57:05.735 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd4f40a1480 0x7fd4f40a3940 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.735 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd508102010 0x7fd50819dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.735 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd508100480 0x7fd508196550 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.735 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 --2- 172.21.15.42:0/129424762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd508101670 0x7fd508196a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.735 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 >> 172.21.15.42:0/129424762 conn(0x7fd5080fb9f0 msgr2=0x7fd508104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:05.735 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 shutdown_connections 2023-12-12T09:57:05.735 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.731+0000 7fd50fb44700 1 -- 172.21.15.42:0/129424762 wait complete. 2023-12-12T09:57:05.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-12-12T09:57:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:05 smithi173 ceph-mon[103308]: pgmap v275: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-12T09:57:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:05 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1834803000' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-12T09:57:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:05 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1834803000' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-12T09:57:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:05 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] up:active 2023-12-12T09:57:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:05 smithi173 ceph-mon[103308]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-12T09:57:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:05 smithi173 ceph-mon[103308]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-12-12T09:57:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:05 smithi173 ceph-mon[103308]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2023-12-12T09:57:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:05 smithi173 ceph-mon[103308]: daemon mds.h is now active in filesystem cephfs as rank 3 2023-12-12T09:57:05.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.872+0000 7efd874bc700 1 -- 172.21.15.42:0/1481984661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd80102b60 msgr2=0x7efd80102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.872+0000 7efd874bc700 1 --2- 172.21.15.42:0/1481984661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd80102b60 0x7efd80102f80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0099c0 tx=0x7efd7c009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.872+0000 7efd874bc700 1 -- 172.21.15.42:0/1481984661 shutdown_connections 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.872+0000 7efd874bc700 1 --2- 172.21.15.42:0/1481984661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd800fff70 0x7efd80109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.872+0000 7efd874bc700 1 --2- 172.21.15.42:0/1481984661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd80102b60 0x7efd80102f80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.872+0000 7efd874bc700 1 --2- 172.21.15.42:0/1481984661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd800ff5d0 0x7efd800ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.872+0000 7efd874bc700 1 -- 172.21.15.42:0/1481984661 >> 172.21.15.42:0/1481984661 conn(0x7efd800fb170 msgr2=0x7efd800fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.872+0000 7efd874bc700 1 -- 172.21.15.42:0/1481984661 shutdown_connections 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.872+0000 7efd874bc700 1 -- 172.21.15.42:0/1481984661 wait complete. 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd874bc700 1 Processor -- start 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd874bc700 1 -- start start 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd874bc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd800ff5d0 0x7efd80199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd874bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd800fff70 0x7efd80199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd874bc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd80102b60 0x7efd801a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd874bc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd801a1260 con 0x7efd800fff70 2023-12-12T09:57:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd874bc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd801a13a0 con 0x7efd800ff5d0 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd874bc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd801a1530 con 0x7efd80102b60 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd84a57700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd800fff70 0x7efd80199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd84a57700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd800fff70 0x7efd80199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33912/0 (socket says 172.21.15.42:33912) 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd84a57700 1 -- 172.21.15.42:0/1089626270 learned_addr learned my addr 172.21.15.42:0/1089626270 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd85258700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd800ff5d0 0x7efd80199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd85a59700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd80102b60 0x7efd801a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd84a57700 1 -- 172.21.15.42:0/1089626270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd80102b60 msgr2=0x7efd801a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd84a57700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd80102b60 0x7efd801a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd84a57700 1 -- 172.21.15.42:0/1089626270 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd800ff5d0 msgr2=0x7efd80199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd84a57700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd800ff5d0 0x7efd80199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.873+0000 7efd84a57700 1 -- 172.21.15.42:0/1089626270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c0095f0 con 0x7efd800fff70 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.874+0000 7efd84a57700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd800fff70 0x7efd80199c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd7400c920 tx=0x7efd7400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.874+0000 7efd727fc700 1 -- 172.21.15.42:0/1089626270 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74007b00 con 0x7efd800fff70 2023-12-12T09:57:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.874+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd801a1810 con 0x7efd800fff70 2023-12-12T09:57:05.879 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.874+0000 7efd727fc700 1 -- 172.21.15.42:0/1089626270 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd7400ce70 con 0x7efd800fff70 2023-12-12T09:57:05.879 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.874+0000 7efd727fc700 1 -- 172.21.15.42:0/1089626270 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74005460 con 0x7efd800fff70 2023-12-12T09:57:05.879 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.874+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd801a1d70 con 0x7efd800fff70 2023-12-12T09:57:05.879 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.875+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd80067a80 con 0x7efd800fff70 2023-12-12T09:57:05.879 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.876+0000 7efd727fc700 1 -- 172.21.15.42:0/1089626270 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7efd740055c0 con 0x7efd800fff70 2023-12-12T09:57:05.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.876+0000 7efd727fc700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efd6c0a1480 0x7efd6c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:05.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.877+0000 7efd727fc700 1 -- 172.21.15.42:0/1089626270 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efd740ccf00 con 0x7efd800fff70 2023-12-12T09:57:05.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.877+0000 7efd85258700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efd6c0a1480 0x7efd6c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:05.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.877+0000 7efd85258700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efd6c0a1480 0x7efd6c0a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efd7c003820 tx=0x7efd7c005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:05.882 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:05.880+0000 7efd727fc700 1 -- 172.21.15.42:0/1089626270 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7efd74096370 con 0x7efd800fff70 2023-12-12T09:57:05.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:57:05.686+0000 7f8df1b7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:57:05.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:57:05.686+0000 7f8df1b7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:57:05.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:57:05.686+0000 7f8df1b7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:57:05.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:57:05.686+0000 7f8df1b7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:57:05.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:57:05 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:57:05.686+0000 7f8df1b7f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:57:05.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:05 smithi111 ceph-mon[102565]: pgmap v275: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2023-12-12T09:57:05.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:05 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1834803000' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-12T09:57:05.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:05 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1834803000' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-12T09:57:05.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:05 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] up:active 2023-12-12T09:57:05.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:05 smithi111 ceph-mon[102565]: daemon mds.h assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2023-12-12T09:57:05.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:05 smithi111 ceph-mon[102565]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=c=up:active} 9 up:standby 2023-12-12T09:57:05.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:05 smithi111 ceph-mon[102565]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:creating} 8 up:standby 2023-12-12T09:57:05.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:05 smithi111 ceph-mon[102565]: daemon mds.h is now active in filesystem cephfs as rank 3 2023-12-12T09:57:06.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.117+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7efd801097e0 con 0x7efd6c0a1480 2023-12-12T09:57:06.123 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.122+0000 7efd727fc700 1 -- 172.21.15.42:0/1089626270 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7efd801097e0 con 0x7efd6c0a1480 2023-12-12T09:57:06.124 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv_0/25d4182c-c2b7-4d78-b5d7-ad6213d24559 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efd6c0a1480 msgr2=0x7efd6c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efd6c0a1480 0x7efd6c0a3940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efd7c003820 tx=0x7efd7c005920 comp rx=0 tx=0).stop 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd800fff70 msgr2=0x7efd80199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd800fff70 0x7efd80199c70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7efd7400c920 tx=0x7efd7400cc30 comp rx=0 tx=0).stop 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 shutdown_connections 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efd6c0a1480 0x7efd6c0a3940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd800ff5d0 0x7efd80199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd80102b60 0x7efd801a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 --2- 172.21.15.42:0/1089626270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd800fff70 0x7efd80199c70 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 >> 172.21.15.42:0/1089626270 conn(0x7efd800fb170 msgr2=0x7efd801080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:06.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 shutdown_connections 2023-12-12T09:57:06.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.124+0000 7efd874bc700 1 -- 172.21.15.42:0/1089626270 wait complete. 2023-12-12T09:57:06.138 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.265+0000 7f5959290700 1 -- 172.21.15.42:0/1573383343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954073fb0 msgr2=0x7f5954074430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.265+0000 7f5959290700 1 --2- 172.21.15.42:0/1573383343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954073fb0 0x7f5954074430 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f593c009990 tx=0x7f593c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.265+0000 7f5959290700 1 -- 172.21.15.42:0/1573383343 shutdown_connections 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.265+0000 7f5959290700 1 --2- 172.21.15.42:0/1573383343 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954075590 0x7f59540739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.265+0000 7f5959290700 1 --2- 172.21.15.42:0/1573383343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954109490 0x7f595410b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.265+0000 7f5959290700 1 --2- 172.21.15.42:0/1573383343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954073fb0 0x7f5954074430 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.265+0000 7f5959290700 1 -- 172.21.15.42:0/1573383343 >> 172.21.15.42:0/1573383343 conn(0x7f59540fcb10 msgr2=0x7f59540fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.265+0000 7f5959290700 1 -- 172.21.15.42:0/1573383343 shutdown_connections 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.265+0000 7f5959290700 1 -- 172.21.15.42:0/1573383343 wait complete. 2023-12-12T09:57:06.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f5959290700 1 Processor -- start 2023-12-12T09:57:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f5959290700 1 -- start start 2023-12-12T09:57:06.270 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f5959290700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954073fb0 0x7f5954199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f5959290700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954075590 0x7f5954199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f5959290700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954109490 0x7f59541a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f5959290700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541a1260 con 0x7f5954109490 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f5959290700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541a13a0 con 0x7f5954075590 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f5959290700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541a1530 con 0x7f5954073fb0 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f59537fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954109490 0x7f59541a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f59537fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954109490 0x7f59541a0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33928/0 (socket says 172.21.15.42:33928) 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f59537fe700 1 -- 172.21.15.42:0/3082951054 learned_addr learned my addr 172.21.15.42:0/3082951054 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.266+0000 7f59527fc700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954075590 0x7f5954199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f5952ffd700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954073fb0 0x7f5954199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f59537fe700 1 -- 172.21.15.42:0/3082951054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954073fb0 msgr2=0x7f5954199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f59537fe700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954073fb0 0x7f5954199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f59537fe700 1 -- 172.21.15.42:0/3082951054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954075590 msgr2=0x7f5954199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f59537fe700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954075590 0x7f5954199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f59537fe700 1 -- 172.21.15.42:0/3082951054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c0095f0 con 0x7f5954109490 2023-12-12T09:57:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f59537fe700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954109490 0x7f59541a0c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f594c007ee0 tx=0x7f594c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f594bfff700 1 -- 172.21.15.42:0/3082951054 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c017070 con 0x7f5954109490 2023-12-12T09:57:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f594bfff700 1 -- 172.21.15.42:0/3082951054 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f594c00f040 con 0x7f5954109490 2023-12-12T09:57:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59541a1810 con 0x7f5954109490 2023-12-12T09:57:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f594bfff700 1 -- 172.21.15.42:0/3082951054 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c005550 con 0x7f5954109490 2023-12-12T09:57:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.267+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59541a1d20 con 0x7f5954109490 2023-12-12T09:57:06.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.268+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59541082a0 con 0x7f5954109490 2023-12-12T09:57:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.269+0000 7f594bfff700 1 -- 172.21.15.42:0/3082951054 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) v1 ==== 136006+0+0 (secure 0 0 0) 0x7f594c00db20 con 0x7f5954109490 2023-12-12T09:57:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.270+0000 7f594bfff700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f59400a1480 0x7f59400a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.270+0000 7f594bfff700 1 -- 172.21.15.42:0/3082951054 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f594c0cc5f0 con 0x7f5954109490 2023-12-12T09:57:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.270+0000 7f5952ffd700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f59400a1480 0x7f59400a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:06.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.270+0000 7f5952ffd700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f59400a1480 0x7f59400a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5944005950 tx=0x7f594400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:06.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.274+0000 7f594bfff700 1 -- 172.21.15.42:0/3082951054 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f594c095a60 con 0x7f5954109490 2023-12-12T09:57:06.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.519+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f5954061a10 con 0x7f59400a1480 2023-12-12T09:57:06.546 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.544+0000 7f594bfff700 1 -- 172.21.15.42:0/3082951054 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5954061a10 con 0x7f59400a1480 2023-12-12T09:57:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f59400a1480 msgr2=0x7f59400a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f59400a1480 0x7f59400a3940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5944005950 tx=0x7f594400b410 comp rx=0 tx=0).stop 2023-12-12T09:57:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954109490 msgr2=0x7f59541a0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954109490 0x7f59541a0c90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f594c007ee0 tx=0x7f594c00d3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 shutdown_connections 2023-12-12T09:57:06.550 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f59400a1480 0x7f59400a3940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5954109490 0x7f59541a0c90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5954073fb0 0x7f5954199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 --2- 172.21.15.42:0/3082951054 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5954075590 0x7f5954199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 >> 172.21.15.42:0/3082951054 conn(0x7f59540fcb10 msgr2=0x7f59541053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 shutdown_connections 2023-12-12T09:57:06.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.546+0000 7f5959290700 1 -- 172.21.15.42:0/3082951054 wait complete. 2023-12-12T09:57:06.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-12-12T09:57:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.693+0000 7f239a544700 1 -- 172.21.15.42:0/2363578723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394100480 msgr2=0x7f23941008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.693+0000 7f239a544700 1 --2- 172.21.15.42:0/2363578723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394100480 0x7f23941008a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f23840099c0 tx=0x7f2384009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.693+0000 7f239a544700 1 -- 172.21.15.42:0/2363578723 shutdown_connections 2023-12-12T09:57:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.693+0000 7f239a544700 1 --2- 172.21.15.42:0/2363578723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394102010 0x7f23941088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.693+0000 7f239a544700 1 --2- 172.21.15.42:0/2363578723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394100480 0x7f23941008a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.693+0000 7f239a544700 1 --2- 172.21.15.42:0/2363578723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2394101670 0x7f2394101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.693+0000 7f239a544700 1 -- 172.21.15.42:0/2363578723 >> 172.21.15.42:0/2363578723 conn(0x7f23940fb9f0 msgr2=0x7f23940fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:06.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.693+0000 7f239a544700 1 -- 172.21.15.42:0/2363578723 shutdown_connections 2023-12-12T09:57:06.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.693+0000 7f239a544700 1 -- 172.21.15.42:0/2363578723 wait complete. 2023-12-12T09:57:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f239a544700 1 Processor -- start 2023-12-12T09:57:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f239a544700 1 -- start start 2023-12-12T09:57:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f239a544700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2394100480 0x7f2394196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f239a544700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394101670 0x7f23941969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f239a544700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394102010 0x7f239419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f239a544700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239419dfb0 con 0x7f2394101670 2023-12-12T09:57:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f239a544700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239419e0f0 con 0x7f2394100480 2023-12-12T09:57:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f239a544700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239419e280 con 0x7f2394102010 2023-12-12T09:57:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f23937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394101670 0x7f23941969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f23937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394101670 0x7f23941969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33942/0 (socket says 172.21.15.42:33942) 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f23937fe700 1 -- 172.21.15.42:0/3083904716 learned_addr learned my addr 172.21.15.42:0/3083904716 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f2393fff700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2394100480 0x7f2394196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f23937fe700 1 -- 172.21.15.42:0/3083904716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394102010 msgr2=0x7f239419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f23937fe700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394102010 0x7f239419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f23937fe700 1 -- 172.21.15.42:0/3083904716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2394100480 msgr2=0x7f2394196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.694+0000 7f23937fe700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2394100480 0x7f2394196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.695+0000 7f23937fe700 1 -- 172.21.15.42:0/3083904716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840095f0 con 0x7f2394101670 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.695+0000 7f23937fe700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394101670 0x7f23941969c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2388009970 tx=0x7f2388009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.695+0000 7f23917fa700 1 -- 172.21.15.42:0/3083904716 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388012070 con 0x7f2394101670 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.695+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2394068670 con 0x7f2394101670 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.695+0000 7f23917fa700 1 -- 172.21.15.42:0/3083904716 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f238800d260 con 0x7f2394101670 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.695+0000 7f23917fa700 1 -- 172.21.15.42:0/3083904716 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388017a60 con 0x7f2394101670 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.695+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2394068b80 con 0x7f2394101670 2023-12-12T09:57:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.696+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2394068310 con 0x7f2394101670 2023-12-12T09:57:06.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.697+0000 7f23917fa700 1 -- 172.21.15.42:0/3083904716 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f23880076b0 con 0x7f2394101670 2023-12-12T09:57:06.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.697+0000 7f23917fa700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f237c0a1510 0x7f237c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:06.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.698+0000 7f2393fff700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f237c0a1510 0x7f237c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:06.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.698+0000 7f23917fa700 1 -- 172.21.15.42:0/3083904716 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f23880ce4e0 con 0x7f2394101670 2023-12-12T09:57:06.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.698+0000 7f2393fff700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f237c0a1510 0x7f237c0a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f23840036f0 tx=0x7f2384005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:06.704 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.702+0000 7f23917fa700 1 -- 172.21.15.42:0/3083904716 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2388097790 con 0x7f2394101670 2023-12-12T09:57:06.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.940+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2394107440 con 0x7f237c0a1510 2023-12-12T09:57:06.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.945+0000 7f23917fa700 1 -- 172.21.15.42:0/3083904716 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f2394107440 con 0x7f237c0a1510 2023-12-12T09:57:06.947 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/_nogroup/sv_1/8a17282b-a5de-4add-a6b2-6f3029871f8b 2023-12-12T09:57:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f237c0a1510 msgr2=0x7f237c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f237c0a1510 0x7f237c0a39d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f23840036f0 tx=0x7f2384005fb0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394101670 msgr2=0x7f23941969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394101670 0x7f23941969c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2388009970 tx=0x7f2388009d30 comp rx=0 tx=0).stop 2023-12-12T09:57:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 shutdown_connections 2023-12-12T09:57:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f237c0a1510 0x7f237c0a39d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2394100480 0x7f2394196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2394102010 0x7f239419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.949 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 --2- 172.21.15.42:0/3083904716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2394101670 0x7f23941969c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:06.950 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 >> 172.21.15.42:0/3083904716 conn(0x7f23940fb9f0 msgr2=0x7f2394104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:06.950 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 shutdown_connections 2023-12-12T09:57:06.950 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:06.947+0000 7f239a544700 1 -- 172.21.15.42:0/3083904716 wait complete. 2023-12-12T09:57:06.961 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:57:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.084+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2790588677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681003b0 msgr2=0x7fee681007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.084+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2790588677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681003b0 0x7fee681007d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee640099c0 tx=0x7fee64009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.084+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2790588677 shutdown_connections 2023-12-12T09:57:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.085+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2790588677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee68101f40 0x7fee681087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.085+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2790588677 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681003b0 0x7fee681007d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.085+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2790588677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee681015a0 0x7fee68101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.085+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2790588677 >> 172.21.15.42:0/2790588677 conn(0x7fee680fb920 msgr2=0x7fee680fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.085+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2790588677 shutdown_connections 2023-12-12T09:57:07.087 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.085+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2790588677 wait complete. 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.085+0000 7fee6f4a8700 1 Processor -- start 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.085+0000 7fee6f4a8700 1 -- start start 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6f4a8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee681003b0 0x7fee68196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6f4a8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681015a0 0x7fee68196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6f4a8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee68101f40 0x7fee6819d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6f4a8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6819de30 con 0x7fee681015a0 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6f4a8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6819df70 con 0x7fee68101f40 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6f4a8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee6819e100 con 0x7fee681003b0 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6ca43700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681015a0 0x7fee68196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6ca43700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681015a0 0x7fee68196840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33956/0 (socket says 172.21.15.42:33956) 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6ca43700 1 -- 172.21.15.42:0/2381901367 learned_addr learned my addr 172.21.15.42:0/2381901367 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6da45700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee68101f40 0x7fee6819d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6d244700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee681003b0 0x7fee68196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6ca43700 1 -- 172.21.15.42:0/2381901367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee681003b0 msgr2=0x7fee68196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6ca43700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee681003b0 0x7fee68196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6ca43700 1 -- 172.21.15.42:0/2381901367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee68101f40 msgr2=0x7fee6819d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6ca43700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee68101f40 0x7fee6819d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6ca43700 1 -- 172.21.15.42:0/2381901367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640095f0 con 0x7fee681015a0 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6d244700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee681003b0 0x7fee68196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee6ca43700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681015a0 0x7fee68196840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c970 tx=0x7fee5800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.086+0000 7fee5e7fc700 1 -- 172.21.15.42:0/2381901367 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58007aa0 con 0x7fee681015a0 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.087+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee6819e3e0 con 0x7fee681015a0 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.087+0000 7fee5e7fc700 1 -- 172.21.15.42:0/2381901367 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee58004cf0 con 0x7fee681015a0 2023-12-12T09:57:07.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.087+0000 7fee5e7fc700 1 -- 172.21.15.42:0/2381901367 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58005600 con 0x7fee681015a0 2023-12-12T09:57:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.087+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee6819e8f0 con 0x7fee681015a0 2023-12-12T09:57:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.087+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee68107760 con 0x7fee681015a0 2023-12-12T09:57:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.088+0000 7fee5e7fc700 1 -- 172.21.15.42:0/2381901367 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fee58004750 con 0x7fee681015a0 2023-12-12T09:57:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.089+0000 7fee5e7fc700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fee540a1510 0x7fee540a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.092 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.089+0000 7fee6d244700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fee540a1510 0x7fee540a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.089+0000 7fee5e7fc700 1 -- 172.21.15.42:0/2381901367 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fee58013070 con 0x7fee681015a0 2023-12-12T09:57:07.093 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.090+0000 7fee6d244700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fee540a1510 0x7fee540a39d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee64003820 tx=0x7fee64000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:07.095 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.093+0000 7fee5e7fc700 1 -- 172.21.15.42:0/2381901367 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fee58096240 con 0x7fee681015a0 2023-12-12T09:57:07.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.345+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee6810bc40 con 0x7fee681015a0 2023-12-12T09:57:07.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.346+0000 7fee5e7fc700 1 -- 172.21.15.42:0/2381901367 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11381 (secure 0 0 0) 0x7fee58004e60 con 0x7fee681015a0 2023-12-12T09:57:07.347 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:07.348 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T09:57:07.293702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24487,"mds_3":24482,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:57:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fee540a1510 msgr2=0x7fee540a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fee540a1510 0x7fee540a39d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fee64003820 tx=0x7fee64000bc0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681015a0 msgr2=0x7fee68196840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681015a0 0x7fee68196840 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee5800c970 tx=0x7fee5800cd30 comp rx=0 tx=0).stop 2023-12-12T09:57:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 shutdown_connections 2023-12-12T09:57:07.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fee540a1510 0x7fee540a39d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fee68101f40 0x7fee6819d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fee681003b0 0x7fee68196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 --2- 172.21.15.42:0/2381901367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fee681015a0 0x7fee68196840 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 >> 172.21.15.42:0/2381901367 conn(0x7fee680fb920 msgr2=0x7fee681048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 shutdown_connections 2023-12-12T09:57:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.348+0000 7fee6f4a8700 1 -- 172.21.15.42:0/2381901367 wait complete. 2023-12-12T09:57:07.351 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2023-12-12T09:57:07.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1789840032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101570 msgr2=0x7fc9a01019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1789840032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101570 0x7fc9a01019d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc9900099c0 tx=0x7fc990009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1789840032 shutdown_connections 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1789840032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9a0100380 0x7fc9a01007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1789840032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0101f10 0x7fc9a01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1789840032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101570 0x7fc9a01019d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1789840032 >> 172.21.15.42:0/1789840032 conn(0x7fc9a00fb8f0 msgr2=0x7fc9a00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1789840032 shutdown_connections 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1789840032 wait complete. 2023-12-12T09:57:07.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 Processor -- start 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.487+0000 7fc9a65f4700 1 -- start start 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a65f4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9a0100380 0x7fc9a0196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a65f4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0101570 0x7fc9a0196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a65f4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101f10 0x7fc9a019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a65f4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a019de60 con 0x7fc9a0101f10 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a65f4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a019dfa0 con 0x7fc9a0100380 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a65f4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a019e130 con 0x7fc9a0101570 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a4b91700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101f10 0x7fc9a019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a4b91700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101f10 0x7fc9a019d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33994/0 (socket says 172.21.15.42:33994) 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a4b91700 1 -- 172.21.15.42:0/1696753927 learned_addr learned my addr 172.21.15.42:0/1696753927 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc99ffff700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9a0100380 0x7fc9a0196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc99f7fe700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0101570 0x7fc9a0196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a4b91700 1 -- 172.21.15.42:0/1696753927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0101570 msgr2=0x7fc9a0196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a4b91700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0101570 0x7fc9a0196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a4b91700 1 -- 172.21.15.42:0/1696753927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9a0100380 msgr2=0x7fc9a0196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a4b91700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9a0100380 0x7fc9a0196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a4b91700 1 -- 172.21.15.42:0/1696753927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9900095f0 con 0x7fc9a0101f10 2023-12-12T09:57:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a4b91700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101f10 0x7fc9a019d890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc99800ef30 tx=0x7fc99800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc99d7fa700 1 -- 172.21.15.42:0/1696753927 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998018070 con 0x7fc9a0101f10 2023-12-12T09:57:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a019e3b0 con 0x7fc9a0101f10 2023-12-12T09:57:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc99d7fa700 1 -- 172.21.15.42:0/1696753927 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc99800f040 con 0x7fc9a0101f10 2023-12-12T09:57:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.488+0000 7fc99d7fa700 1 -- 172.21.15.42:0/1696753927 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99800cd00 con 0x7fc9a0101f10 2023-12-12T09:57:07.493 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.489+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a019e870 con 0x7fc9a0101f10 2023-12-12T09:57:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.490+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a0107800 con 0x7fc9a0101f10 2023-12-12T09:57:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.491+0000 7fc99d7fa700 1 -- 172.21.15.42:0/1696753927 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fc998003a00 con 0x7fc9a0101f10 2023-12-12T09:57:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.491+0000 7fc99d7fa700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc9880a1510 0x7fc9880a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.491+0000 7fc99d7fa700 1 -- 172.21.15.42:0/1696753927 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fc9980ccbf0 con 0x7fc9a0101f10 2023-12-12T09:57:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.491+0000 7fc99ffff700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc9880a1510 0x7fc9880a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.494 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.492+0000 7fc99ffff700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc9880a1510 0x7fc9880a39d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc9a01013d0 tx=0x7fc99400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:07.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.496+0000 7fc99d7fa700 1 -- 172.21.15.42:0/1696753927 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc998095f90 con 0x7fc9a0101f10 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: pgmap v276: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] up:active 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: Cluster is now healthy 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:07.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:07 smithi173 ceph-mon[103308]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-12-12T09:57:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: pgmap v276: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2023-12-12T09:57:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] up:active 2023-12-12T09:57:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-12T09:57:07.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-12T09:57:07.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: Cluster is now healthy 2023-12-12T09:57:07.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2023-12-12T09:57:07.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-12-12T09:57:07.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-12-12T09:57:07.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:07.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:07 smithi042 ceph-mon[98894]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-12-12T09:57:07.706 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.704+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9a010bbf0 con 0x7fc9a0101f10 2023-12-12T09:57:07.707 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.705+0000 7fc99d7fa700 1 -- 172.21.15.42:0/1696753927 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7fc9980995b0 con 0x7fc9a0101f10 2023-12-12T09:57:07.708 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:07.708 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.707+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc9880a1510 msgr2=0x7fc9880a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.707+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc9880a1510 0x7fc9880a39d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc9a01013d0 tx=0x7fc99400b410 comp rx=0 tx=0).stop 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.707+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101f10 msgr2=0x7fc9a019d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.707+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101f10 0x7fc9a019d890 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc99800ef30 tx=0x7fc99800c3a0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.707+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 shutdown_connections 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.707+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc9880a1510 0x7fc9880a39d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.707+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc9a0100380 0x7fc9a0196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.707+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9a0101f10 0x7fc9a019d890 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.707+0000 7fc9a65f4700 1 --2- 172.21.15.42:0/1696753927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a0101570 0x7fc9a0196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.708+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 >> 172.21.15.42:0/1696753927 conn(0x7fc9a00fb8f0 msgr2=0x7fc9a0104940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.708+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 shutdown_connections 2023-12-12T09:57:07.710 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.708+0000 7fc9a65f4700 1 -- 172.21.15.42:0/1696753927 wait complete. 2023-12-12T09:57:07.724 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-12T09:57:07.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:57:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: from='client.14880 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: pgmap v276: 129 pgs: 129 active+clean; 581 KiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2023-12-12T09:57:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: from='client.14892 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] up:active 2023-12-12T09:57:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2023-12-12T09:57:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-12T09:57:07.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: Cluster is now healthy 2023-12-12T09:57:07.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=c=up:active,3=h=up:active} 8 up:standby 2023-12-12T09:57:07.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2023-12-12T09:57:07.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: daemon mds.f is now active in filesystem cephfs as rank 4 2023-12-12T09:57:07.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: from='client.14898 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:07.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:07 smithi111 ceph-mon[102565]: mgrmap e17: x(active, since 6m), standbys: z, y 2023-12-12T09:57:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 -- 172.21.15.42:0/390055730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e001071e0 msgr2=0x7f1e001095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 --2- 172.21.15.42:0/390055730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e001071e0 0x7f1e001095d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1df800b0a0 tx=0x7f1df800b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 -- 172.21.15.42:0/390055730 shutdown_connections 2023-12-12T09:57:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 --2- 172.21.15.42:0/390055730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e001071e0 0x7f1e001095d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 --2- 172.21.15.42:0/390055730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e00101f80 0x7f1e00104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 --2- 172.21.15.42:0/390055730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e001048b0 0x7f1e00106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 -- 172.21.15.42:0/390055730 >> 172.21.15.42:0/390055730 conn(0x7f1e000fb8f0 msgr2=0x7f1e000fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 -- 172.21.15.42:0/390055730 shutdown_connections 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 -- 172.21.15.42:0/390055730 wait complete. 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.848+0000 7f1e055fb700 1 Processor -- start 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1e055fb700 1 -- start start 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1e055fb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e00101f80 0x7f1e00101850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1e055fb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e001048b0 0x7f1e000fff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1e055fb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e001071e0 0x7f1e00100470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1e055fb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00100b70 con 0x7f1e00101f80 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1e055fb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e00100ce0 con 0x7f1e001071e0 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1e055fb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e001a1590 con 0x7f1e001048b0 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e00101f80 0x7f1e00101850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dfeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e00101f80 0x7f1e00101850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34004/0 (socket says 172.21.15.42:34004) 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dfeffd700 1 -- 172.21.15.42:0/1637761820 learned_addr learned my addr 172.21.15.42:0/1637761820 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:07.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dff7fe700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e001071e0 0x7f1e00100470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dfe7fc700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e001048b0 0x7f1e000fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dfeffd700 1 -- 172.21.15.42:0/1637761820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e001048b0 msgr2=0x7f1e000fff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dfeffd700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e001048b0 0x7f1e000fff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dfeffd700 1 -- 172.21.15.42:0/1637761820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e001071e0 msgr2=0x7f1e00100470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dfeffd700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e001071e0 0x7f1e00100470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dfeffd700 1 -- 172.21.15.42:0/1637761820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009d00 con 0x7f1e00101f80 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.849+0000 7f1dff7fe700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e001071e0 0x7f1e00100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.850+0000 7f1dfeffd700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e00101f80 0x7f1e00101850 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1df000e9a0 tx=0x7f1df000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.850+0000 7f1df7fff700 1 -- 172.21.15.42:0/1637761820 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0009790 con 0x7f1e00101f80 2023-12-12T09:57:07.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.850+0000 7f1df7fff700 1 -- 172.21.15.42:0/1637761820 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1df0010430 con 0x7f1e00101f80 2023-12-12T09:57:07.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.850+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e001a17f0 con 0x7f1e00101f80 2023-12-12T09:57:07.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.850+0000 7f1df7fff700 1 -- 172.21.15.42:0/1637761820 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df0018680 con 0x7f1e00101f80 2023-12-12T09:57:07.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.850+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e001a1d00 con 0x7f1e00101f80 2023-12-12T09:57:07.856 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.851+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e000fd470 con 0x7f1e00101f80 2023-12-12T09:57:07.856 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.852+0000 7f1df7fff700 1 -- 172.21.15.42:0/1637761820 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f1df00105a0 con 0x7f1e00101f80 2023-12-12T09:57:07.856 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.853+0000 7f1df7fff700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1dec0a1510 0x7f1dec0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:07.856 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.853+0000 7f1dfe7fc700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1dec0a1510 0x7f1dec0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:07.856 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.853+0000 7f1df7fff700 1 -- 172.21.15.42:0/1637761820 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1df0014070 con 0x7f1e00101f80 2023-12-12T09:57:07.856 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.853+0000 7f1dfe7fc700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1dec0a1510 0x7f1dec0a39d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009c60 tx=0x7f1de8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:07.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:07.857+0000 7f1df7fff700 1 -- 172.21.15.42:0/1637761820 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f1df0096730 con 0x7f1e00101f80 2023-12-12T09:57:08.056 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.054+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e0002d3b0 con 0x7f1dec0a1510 2023-12-12T09:57:08.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.070+0000 7f1df7fff700 1 -- 172.21.15.42:0/1637761820 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423083 (secure 0 0 0) 0x7f1e0002d3b0 con 0x7f1dec0a1510 2023-12-12T09:57:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1dec0a1510 msgr2=0x7f1dec0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1dec0a1510 0x7f1dec0a39d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009c60 tx=0x7f1de8009260 comp rx=0 tx=0).stop 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e00101f80 msgr2=0x7f1e00101850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e00101f80 0x7f1e00101850 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1df000e9a0 tx=0x7f1df000ecb0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 shutdown_connections 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f1dec0a1510 0x7f1dec0a39d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e001071e0 0x7f1e00100470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e00101f80 0x7f1e00101850 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 --2- 172.21.15.42:0/1637761820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e001048b0 0x7f1e000fff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 >> 172.21.15.42:0/1637761820 conn(0x7f1e000fb8f0 msgr2=0x7f1e001028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 shutdown_connections 2023-12-12T09:57:08.145 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.074+0000 7f1e055fb700 1 -- 172.21.15.42:0/1637761820 wait complete. 2023-12-12T09:57:08.146 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2023-12-12T09:57:08.152 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2023-12-12T09:57:07.814046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600649,"num_objects":80,"num_object_clones":0,"num_object_copies":241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":152,"num_read_kb":134,"num_write":345,"num_write_kb":2997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":284,"ondisk_log_size":284,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337432,"kb_used_data":13236,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604392,"statfs":{"total":1151940427776,"available":1151594897408,"internally_reserved":0,"allocated":13553664,"data_stored":7408787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5784,"num_objects":56,"num_object_clones":0,"num_object_copies":169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":117,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003719"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494114+0000","last_change":"2023-12-12T09:56:38.273952+0000","last_active":"2023-12-12T09:56:42.494114+0000","last_peered":"2023-12-12T09:56:42.494114+0000","last_clean":"2023-12-12T09:56:42.494114+0000","last_became_active":"2023-12-12T09:56:38.273609+0000","last_became_peered":"2023-12-12T09:56:38.273609+0000","last_unstale":"2023-12-12T09:56:42.494114+0000","last_undegraded":"2023-12-12T09:56:42.494114+0000","last_fullsized":"2023-12-12T09:56:42.494114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:23:43.144987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496587+0000","last_change":"2023-12-12T09:56:38.278719+0000","last_active":"2023-12-12T09:56:42.496587+0000","last_peered":"2023-12-12T09:56:42.496587+0000","last_clean":"2023-12-12T09:56:42.496587+0000","last_became_active":"2023-12-12T09:56:38.278404+0000","last_became_peered":"2023-12-12T09:56:38.278404+0000","last_unstale":"2023-12-12T09:56:42.496587+0000","last_undegraded":"2023-12-12T09:56:42.496587+0000","last_fullsized":"2023-12-12T09:56:42.496587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:34:01.139106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494931+0000","last_change":"2023-12-12T09:56:38.274276+0000","last_active":"2023-12-12T09:56:42.494931+0000","last_peered":"2023-12-12T09:56:42.494931+0000","last_clean":"2023-12-12T09:56:42.494931+0000","last_became_active":"2023-12-12T09:56:38.273974+0000","last_became_peered":"2023-12-12T09:56:38.273974+0000","last_unstale":"2023-12-12T09:56:42.494931+0000","last_undegraded":"2023-12-12T09:56:42.494931+0000","last_fullsized":"2023-12-12T09:56:42.494931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:40:55.979238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.490962+0000","last_change":"2023-12-12T09:56:38.276021+0000","last_active":"2023-12-12T09:56:42.490962+0000","last_peered":"2023-12-12T09:56:42.490962+0000","last_clean":"2023-12-12T09:56:42.490962+0000","last_became_active":"2023-12-12T09:56:38.275305+0000","last_became_peered":"2023-12-12T09:56:38.275305+0000","last_unstale":"2023-12-12T09:56:42.490962+0000","last_undegraded":"2023-12-12T09:56:42.490962+0000","last_fullsized":"2023-12-12T09:56:42.490962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:32:28.839430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.491750+0000","last_change":"2023-12-12T09:56:38.286720+0000","last_active":"2023-12-12T09:56:42.491750+0000","last_peered":"2023-12-12T09:56:42.491750+0000","last_clean":"2023-12-12T09:56:42.491750+0000","last_became_active":"2023-12-12T09:56:38.286410+0000","last_became_peered":"2023-12-12T09:56:38.286410+0000","last_unstale":"2023-12-12T09:56:42.491750+0000","last_undegraded":"2023-12-12T09:56:42.491750+0000","last_fullsized":"2023-12-12T09:56:42.491750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:07:42.131132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576251+0000","last_change":"2023-12-12T09:56:38.273350+0000","last_active":"2023-12-12T09:56:42.576251+0000","last_peered":"2023-12-12T09:56:42.576251+0000","last_clean":"2023-12-12T09:56:42.576251+0000","last_became_active":"2023-12-12T09:56:38.273077+0000","last_became_peered":"2023-12-12T09:56:38.273077+0000","last_unstale":"2023-12-12T09:56:42.576251+0000","last_undegraded":"2023-12-12T09:56:42.576251+0000","last_fullsized":"2023-12-12T09:56:42.576251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:28.081132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496617+0000","last_change":"2023-12-12T09:56:38.278212+0000","last_active":"2023-12-12T09:56:42.496617+0000","last_peered":"2023-12-12T09:56:42.496617+0000","last_clean":"2023-12-12T09:56:42.496617+0000","last_became_active":"2023-12-12T09:56:38.277913+0000","last_became_peered":"2023-12-12T09:56:38.277913+0000","last_unstale":"2023-12-12T09:56:42.496617+0000","last_undegraded":"2023-12-12T09:56:42.496617+0000","last_fullsized":"2023-12-12T09:56:42.496617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:39:11.893381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.486676+0000","last_change":"2023-12-12T09:56:38.286625+0000","last_active":"2023-12-12T09:56:42.486676+0000","last_peered":"2023-12-12T09:56:42.486676+0000","last_clean":"2023-12-12T09:56:42.486676+0000","last_became_active":"2023-12-12T09:56:38.286035+0000","last_became_peered":"2023-12-12T09:56:38.286035+0000","last_unstale":"2023-12-12T09:56:42.486676+0000","last_undegraded":"2023-12-12T09:56:42.486676+0000","last_fullsized":"2023-12-12T09:56:42.486676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:13:03.723529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495016+0000","last_change":"2023-12-12T09:56:38.281533+0000","last_active":"2023-12-12T09:56:42.495016+0000","last_peered":"2023-12-12T09:56:42.495016+0000","last_clean":"2023-12-12T09:56:42.495016+0000","last_became_active":"2023-12-12T09:56:38.281228+0000","last_became_peered":"2023-12-12T09:56:38.281228+0000","last_unstale":"2023-12-12T09:56:42.495016+0000","last_undegraded":"2023-12-12T09:56:42.495016+0000","last_fullsized":"2023-12-12T09:56:42.495016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:49:34.075982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576986+0000","last_change":"2023-12-12T09:56:38.274599+0000","last_active":"2023-12-12T09:56:42.576986+0000","last_peered":"2023-12-12T09:56:42.576986+0000","last_clean":"2023-12-12T09:56:42.576986+0000","last_became_active":"2023-12-12T09:56:38.274309+0000","last_became_peered":"2023-12-12T09:56:38.274309+0000","last_unstale":"2023-12-12T09:56:42.576986+0000","last_undegraded":"2023-12-12T09:56:42.576986+0000","last_fullsized":"2023-12-12T09:56:42.576986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:05:06.054345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497435+0000","last_change":"2023-12-12T09:56:38.279348+0000","last_active":"2023-12-12T09:56:42.497435+0000","last_peered":"2023-12-12T09:56:42.497435+0000","last_clean":"2023-12-12T09:56:42.497435+0000","last_became_active":"2023-12-12T09:56:38.278843+0000","last_became_peered":"2023-12-12T09:56:38.278843+0000","last_unstale":"2023-12-12T09:56:42.497435+0000","last_undegraded":"2023-12-12T09:56:42.497435+0000","last_fullsized":"2023-12-12T09:56:42.497435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:50:04.248976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.490866+0000","last_change":"2023-12-12T09:56:38.276052+0000","last_active":"2023-12-12T09:56:42.490866+0000","last_peered":"2023-12-12T09:56:42.490866+0000","last_clean":"2023-12-12T09:56:42.490866+0000","last_became_active":"2023-12-12T09:56:38.275712+0000","last_became_peered":"2023-12-12T09:56:38.275712+0000","last_unstale":"2023-12-12T09:56:42.490866+0000","last_undegraded":"2023-12-12T09:56:42.490866+0000","last_fullsized":"2023-12-12T09:56:42.490866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:37:06.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499766+0000","last_change":"2023-12-12T09:56:38.284052+0000","last_active":"2023-12-12T09:56:42.499766+0000","last_peered":"2023-12-12T09:56:42.499766+0000","last_clean":"2023-12-12T09:56:42.499766+0000","last_became_active":"2023-12-12T09:56:38.283741+0000","last_became_peered":"2023-12-12T09:56:38.283741+0000","last_unstale":"2023-12-12T09:56:42.499766+0000","last_undegraded":"2023-12-12T09:56:42.499766+0000","last_fullsized":"2023-12-12T09:56:42.499766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:40:47.505972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494574+0000","last_change":"2023-12-12T09:56:38.281604+0000","last_active":"2023-12-12T09:56:42.494574+0000","last_peered":"2023-12-12T09:56:42.494574+0000","last_clean":"2023-12-12T09:56:42.494574+0000","last_became_active":"2023-12-12T09:56:38.281247+0000","last_became_peered":"2023-12-12T09:56:38.281247+0000","last_unstale":"2023-12-12T09:56:42.494574+0000","last_undegraded":"2023-12-12T09:56:42.494574+0000","last_fullsized":"2023-12-12T09:56:42.494574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:36:14.002676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495303+0000","last_change":"2023-12-12T09:56:38.273599+0000","last_active":"2023-12-12T09:56:42.495303+0000","last_peered":"2023-12-12T09:56:42.495303+0000","last_clean":"2023-12-12T09:56:42.495303+0000","last_became_active":"2023-12-12T09:56:38.273290+0000","last_became_peered":"2023-12-12T09:56:38.273290+0000","last_unstale":"2023-12-12T09:56:42.495303+0000","last_undegraded":"2023-12-12T09:56:42.495303+0000","last_fullsized":"2023-12-12T09:56:42.495303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:54:52.088314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495603+0000","last_change":"2023-12-12T09:56:38.275663+0000","last_active":"2023-12-12T09:56:42.495603+0000","last_peered":"2023-12-12T09:56:42.495603+0000","last_clean":"2023-12-12T09:56:42.495603+0000","last_became_active":"2023-12-12T09:56:38.275322+0000","last_became_peered":"2023-12-12T09:56:38.275322+0000","last_unstale":"2023-12-12T09:56:42.495603+0000","last_undegraded":"2023-12-12T09:56:42.495603+0000","last_fullsized":"2023-12-12T09:56:42.495603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:46:50.067328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494023+0000","last_change":"2023-12-12T09:56:38.274064+0000","last_active":"2023-12-12T09:56:42.494023+0000","last_peered":"2023-12-12T09:56:42.494023+0000","last_clean":"2023-12-12T09:56:42.494023+0000","last_became_active":"2023-12-12T09:56:38.273733+0000","last_became_peered":"2023-12-12T09:56:38.273733+0000","last_unstale":"2023-12-12T09:56:42.494023+0000","last_undegraded":"2023-12-12T09:56:42.494023+0000","last_fullsized":"2023-12-12T09:56:42.494023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:20:31.662432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.501075+0000","last_change":"2023-12-12T09:56:38.277627+0000","last_active":"2023-12-12T09:56:42.501075+0000","last_peered":"2023-12-12T09:56:42.501075+0000","last_clean":"2023-12-12T09:56:42.501075+0000","last_became_active":"2023-12-12T09:56:38.277275+0000","last_became_peered":"2023-12-12T09:56:38.277275+0000","last_unstale":"2023-12-12T09:56:42.501075+0000","last_undegraded":"2023-12-12T09:56:42.501075+0000","last_fullsized":"2023-12-12T09:56:42.501075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:37:57.256691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496895+0000","last_change":"2023-12-12T09:56:38.276900+0000","last_active":"2023-12-12T09:56:42.496895+0000","last_peered":"2023-12-12T09:56:42.496895+0000","last_clean":"2023-12-12T09:56:42.496895+0000","last_became_active":"2023-12-12T09:56:38.276384+0000","last_became_peered":"2023-12-12T09:56:38.276384+0000","last_unstale":"2023-12-12T09:56:42.496895+0000","last_undegraded":"2023-12-12T09:56:42.496895+0000","last_fullsized":"2023-12-12T09:56:42.496895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:13:37.097349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.554898+0000","last_change":"2023-12-12T09:56:42.509239+0000","last_active":"2023-12-12T09:56:43.554898+0000","last_peered":"2023-12-12T09:56:43.554898+0000","last_clean":"2023-12-12T09:56:43.554898+0000","last_became_active":"2023-12-12T09:56:42.508699+0000","last_became_peered":"2023-12-12T09:56:42.508699+0000","last_unstale":"2023-12-12T09:56:43.554898+0000","last_undegraded":"2023-12-12T09:56:43.554898+0000","last_fullsized":"2023-12-12T09:56:43.554898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:12:39.501023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:04.288334+0000","last_change":"2023-12-12T09:56:37.476258+0000","last_active":"2023-12-12T09:57:04.288334+0000","last_peered":"2023-12-12T09:57:04.288334+0000","last_clean":"2023-12-12T09:57:04.288334+0000","last_became_active":"2023-12-12T09:56:37.475886+0000","last_became_peered":"2023-12-12T09:56:37.475886+0000","last_unstale":"2023-12-12T09:57:04.288334+0000","last_undegraded":"2023-12-12T09:57:04.288334+0000","last_fullsized":"2023-12-12T09:57:04.288334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:29:34.770549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.486999+0000","last_change":"2023-12-12T09:56:38.284968+0000","last_active":"2023-12-12T09:56:42.486999+0000","last_peered":"2023-12-12T09:56:42.486999+0000","last_clean":"2023-12-12T09:56:42.486999+0000","last_became_active":"2023-12-12T09:56:38.284661+0000","last_became_peered":"2023-12-12T09:56:38.284661+0000","last_unstale":"2023-12-12T09:56:42.486999+0000","last_undegraded":"2023-12-12T09:56:42.486999+0000","last_fullsized":"2023-12-12T09:56:42.486999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:27:02.489220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681309+0000","last_change":"2023-12-12T09:56:42.512514+0000","last_active":"2023-12-12T09:56:43.681309+0000","last_peered":"2023-12-12T09:56:43.681309+0000","last_clean":"2023-12-12T09:56:43.681309+0000","last_became_active":"2023-12-12T09:56:42.512217+0000","last_became_peered":"2023-12-12T09:56:42.512217+0000","last_unstale":"2023-12-12T09:56:43.681309+0000","last_undegraded":"2023-12-12T09:56:43.681309+0000","last_fullsized":"2023-12-12T09:56:43.681309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:46:28.422801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680885+0000","last_change":"2023-12-12T09:56:37.471047+0000","last_active":"2023-12-12T09:56:43.680885+0000","last_peered":"2023-12-12T09:56:43.680885+0000","last_clean":"2023-12-12T09:56:43.680885+0000","last_became_active":"2023-12-12T09:56:37.470689+0000","last_became_peered":"2023-12-12T09:56:37.470689+0000","last_unstale":"2023-12-12T09:56:43.680885+0000","last_undegraded":"2023-12-12T09:56:43.680885+0000","last_fullsized":"2023-12-12T09:56:43.680885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:00.648165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496361+0000","last_change":"2023-12-12T09:56:38.275884+0000","last_active":"2023-12-12T09:56:42.496361+0000","last_peered":"2023-12-12T09:56:42.496361+0000","last_clean":"2023-12-12T09:56:42.496361+0000","last_became_active":"2023-12-12T09:56:38.275161+0000","last_became_peered":"2023-12-12T09:56:38.275161+0000","last_unstale":"2023-12-12T09:56:42.496361+0000","last_undegraded":"2023-12-12T09:56:42.496361+0000","last_fullsized":"2023-12-12T09:56:42.496361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:06:02.863719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682109+0000","last_change":"2023-12-12T09:56:42.507539+0000","last_active":"2023-12-12T09:56:43.682109+0000","last_peered":"2023-12-12T09:56:43.682109+0000","last_clean":"2023-12-12T09:56:43.682109+0000","last_became_active":"2023-12-12T09:56:42.507008+0000","last_became_peered":"2023-12-12T09:56:42.507008+0000","last_unstale":"2023-12-12T09:56:43.682109+0000","last_undegraded":"2023-12-12T09:56:43.682109+0000","last_fullsized":"2023-12-12T09:56:43.682109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:28:30.449096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.302249+0000","last_change":"2023-12-12T09:56:37.475002+0000","last_active":"2023-12-12T09:57:06.302249+0000","last_peered":"2023-12-12T09:57:06.302249+0000","last_clean":"2023-12-12T09:57:06.302249+0000","last_became_active":"2023-12-12T09:56:37.474675+0000","last_became_peered":"2023-12-12T09:56:37.474675+0000","last_unstale":"2023-12-12T09:57:06.302249+0000","last_undegraded":"2023-12-12T09:57:06.302249+0000","last_fullsized":"2023-12-12T09:57:06.302249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:38:15.880299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.493588+0000","last_change":"2023-12-12T09:56:38.280532+0000","last_active":"2023-12-12T09:56:42.493588+0000","last_peered":"2023-12-12T09:56:42.493588+0000","last_clean":"2023-12-12T09:56:42.493588+0000","last_became_active":"2023-12-12T09:56:38.280177+0000","last_became_peered":"2023-12-12T09:56:38.280177+0000","last_unstale":"2023-12-12T09:56:42.493588+0000","last_undegraded":"2023-12-12T09:56:42.493588+0000","last_fullsized":"2023-12-12T09:56:42.493588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:02:34.818978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681062+0000","last_change":"2023-12-12T09:56:42.504506+0000","last_active":"2023-12-12T09:56:43.681062+0000","last_peered":"2023-12-12T09:56:43.681062+0000","last_clean":"2023-12-12T09:56:43.681062+0000","last_became_active":"2023-12-12T09:56:42.504104+0000","last_became_peered":"2023-12-12T09:56:42.504104+0000","last_unstale":"2023-12-12T09:56:43.681062+0000","last_undegraded":"2023-12-12T09:56:43.681062+0000","last_fullsized":"2023-12-12T09:56:43.681062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:12:41.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.301432+0000","last_change":"2023-12-12T09:56:37.475222+0000","last_active":"2023-12-12T09:57:06.301432+0000","last_peered":"2023-12-12T09:57:06.301432+0000","last_clean":"2023-12-12T09:57:06.301432+0000","last_became_active":"2023-12-12T09:56:37.474917+0000","last_became_peered":"2023-12-12T09:56:37.474917+0000","last_unstale":"2023-12-12T09:57:06.301432+0000","last_undegraded":"2023-12-12T09:57:06.301432+0000","last_fullsized":"2023-12-12T09:57:06.301432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:26:36.639215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487408+0000","last_change":"2023-12-12T09:56:38.277949+0000","last_active":"2023-12-12T09:56:42.487408+0000","last_peered":"2023-12-12T09:56:42.487408+0000","last_clean":"2023-12-12T09:56:42.487408+0000","last_became_active":"2023-12-12T09:56:38.277625+0000","last_became_peered":"2023-12-12T09:56:38.277625+0000","last_unstale":"2023-12-12T09:56:42.487408+0000","last_undegraded":"2023-12-12T09:56:42.487408+0000","last_fullsized":"2023-12-12T09:56:42.487408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:42:46.906101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.555152+0000","last_change":"2023-12-12T09:56:42.502196+0000","last_active":"2023-12-12T09:56:43.555152+0000","last_peered":"2023-12-12T09:56:43.555152+0000","last_clean":"2023-12-12T09:56:43.555152+0000","last_became_active":"2023-12-12T09:56:42.501789+0000","last_became_peered":"2023-12-12T09:56:42.501789+0000","last_unstale":"2023-12-12T09:56:43.555152+0000","last_undegraded":"2023-12-12T09:56:43.555152+0000","last_fullsized":"2023-12-12T09:56:43.555152+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:04:02.656074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:07.302672+0000","last_change":"2023-12-12T09:56:37.472404+0000","last_active":"2023-12-12T09:57:07.302672+0000","last_peered":"2023-12-12T09:57:07.302672+0000","last_clean":"2023-12-12T09:57:07.302672+0000","last_became_active":"2023-12-12T09:56:37.472063+0000","last_became_peered":"2023-12-12T09:56:37.472063+0000","last_unstale":"2023-12-12T09:57:07.302672+0000","last_undegraded":"2023-12-12T09:57:07.302672+0000","last_fullsized":"2023-12-12T09:57:07.302672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:07.020310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.492300+0000","last_change":"2023-12-12T09:56:38.281152+0000","last_active":"2023-12-12T09:56:42.492300+0000","last_peered":"2023-12-12T09:56:42.492300+0000","last_clean":"2023-12-12T09:56:42.492300+0000","last_became_active":"2023-12-12T09:56:38.280793+0000","last_became_peered":"2023-12-12T09:56:38.280793+0000","last_unstale":"2023-12-12T09:56:42.492300+0000","last_undegraded":"2023-12-12T09:56:42.492300+0000","last_fullsized":"2023-12-12T09:56:42.492300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.554920+0000","last_change":"2023-12-12T09:56:42.507439+0000","last_active":"2023-12-12T09:56:43.554920+0000","last_peered":"2023-12-12T09:56:43.554920+0000","last_clean":"2023-12-12T09:56:43.554920+0000","last_became_active":"2023-12-12T09:56:42.506980+0000","last_became_peered":"2023-12-12T09:56:42.506980+0000","last_unstale":"2023-12-12T09:56:43.554920+0000","last_undegraded":"2023-12-12T09:56:43.554920+0000","last_fullsized":"2023-12-12T09:56:43.554920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:37.086804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576251+0000","last_change":"2023-12-12T09:56:37.473666+0000","last_active":"2023-12-12T09:56:42.576251+0000","last_peered":"2023-12-12T09:56:42.576251+0000","last_clean":"2023-12-12T09:56:42.576251+0000","last_became_active":"2023-12-12T09:56:37.473349+0000","last_became_peered":"2023-12-12T09:56:37.473349+0000","last_unstale":"2023-12-12T09:56:42.576251+0000","last_undegraded":"2023-12-12T09:56:42.576251+0000","last_fullsized":"2023-12-12T09:56:42.576251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.590517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487307+0000","last_change":"2023-12-12T09:56:38.279220+0000","last_active":"2023-12-12T09:56:42.487307+0000","last_peered":"2023-12-12T09:56:42.487307+0000","last_clean":"2023-12-12T09:56:42.487307+0000","last_became_active":"2023-12-12T09:56:38.278950+0000","last_became_peered":"2023-12-12T09:56:38.278950+0000","last_unstale":"2023-12-12T09:56:42.487307+0000","last_undegraded":"2023-12-12T09:56:42.487307+0000","last_fullsized":"2023-12-12T09:56:42.487307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:06:24.247294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680871+0000","last_change":"2023-12-12T09:56:42.507022+0000","last_active":"2023-12-12T09:56:43.680871+0000","last_peered":"2023-12-12T09:56:43.680871+0000","last_clean":"2023-12-12T09:56:43.680871+0000","last_became_active":"2023-12-12T09:56:42.506575+0000","last_became_peered":"2023-12-12T09:56:42.506575+0000","last_unstale":"2023-12-12T09:56:43.680871+0000","last_undegraded":"2023-12-12T09:56:43.680871+0000","last_fullsized":"2023-12-12T09:56:43.680871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:46.685081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499372+0000","last_change":"2023-12-12T09:56:37.464083+0000","last_active":"2023-12-12T09:56:42.499372+0000","last_peered":"2023-12-12T09:56:42.499372+0000","last_clean":"2023-12-12T09:56:42.499372+0000","last_became_active":"2023-12-12T09:56:37.463774+0000","last_became_peered":"2023-12-12T09:56:37.463774+0000","last_unstale":"2023-12-12T09:56:42.499372+0000","last_undegraded":"2023-12-12T09:56:42.499372+0000","last_fullsized":"2023-12-12T09:56:42.499372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:34.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.486804+0000","last_change":"2023-12-12T09:56:38.278729+0000","last_active":"2023-12-12T09:56:42.486804+0000","last_peered":"2023-12-12T09:56:42.486804+0000","last_clean":"2023-12-12T09:56:42.486804+0000","last_became_active":"2023-12-12T09:56:38.278193+0000","last_became_peered":"2023-12-12T09:56:38.278193+0000","last_unstale":"2023-12-12T09:56:42.486804+0000","last_undegraded":"2023-12-12T09:56:42.486804+0000","last_fullsized":"2023-12-12T09:56:42.486804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:20:25.854114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.262476+0000","last_change":"2023-12-12T09:56:42.504228+0000","last_active":"2023-12-12T09:56:43.262476+0000","last_peered":"2023-12-12T09:56:43.262476+0000","last_clean":"2023-12-12T09:56:43.262476+0000","last_became_active":"2023-12-12T09:56:42.503823+0000","last_became_peered":"2023-12-12T09:56:42.503823+0000","last_unstale":"2023-12-12T09:56:43.262476+0000","last_undegraded":"2023-12-12T09:56:43.262476+0000","last_fullsized":"2023-12-12T09:56:43.262476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:54:05.263784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.300013+0000","last_change":"2023-12-12T09:56:37.472108+0000","last_active":"2023-12-12T09:57:06.300013+0000","last_peered":"2023-12-12T09:57:06.300013+0000","last_clean":"2023-12-12T09:57:06.300013+0000","last_became_active":"2023-12-12T09:56:37.471822+0000","last_became_peered":"2023-12-12T09:56:37.471822+0000","last_unstale":"2023-12-12T09:57:06.300013+0000","last_undegraded":"2023-12-12T09:57:06.300013+0000","last_fullsized":"2023-12-12T09:57:06.300013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:19:24.432739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495125+0000","last_change":"2023-12-12T09:56:38.274724+0000","last_active":"2023-12-12T09:56:42.495125+0000","last_peered":"2023-12-12T09:56:42.495125+0000","last_clean":"2023-12-12T09:56:42.495125+0000","last_became_active":"2023-12-12T09:56:38.274383+0000","last_became_peered":"2023-12-12T09:56:38.274383+0000","last_unstale":"2023-12-12T09:56:42.495125+0000","last_undegraded":"2023-12-12T09:56:42.495125+0000","last_fullsized":"2023-12-12T09:56:42.495125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:21:37.512880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680866+0000","last_change":"2023-12-12T09:56:42.505510+0000","last_active":"2023-12-12T09:56:43.680866+0000","last_peered":"2023-12-12T09:56:43.680866+0000","last_clean":"2023-12-12T09:56:43.680866+0000","last_became_active":"2023-12-12T09:56:42.504141+0000","last_became_peered":"2023-12-12T09:56:42.504141+0000","last_unstale":"2023-12-12T09:56:43.680866+0000","last_undegraded":"2023-12-12T09:56:43.680866+0000","last_fullsized":"2023-12-12T09:56:43.680866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:48:00.108207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.279949+0000","last_change":"2023-12-12T09:56:37.466706+0000","last_active":"2023-12-12T09:57:03.279949+0000","last_peered":"2023-12-12T09:57:03.279949+0000","last_clean":"2023-12-12T09:57:03.279949+0000","last_became_active":"2023-12-12T09:56:37.466268+0000","last_became_peered":"2023-12-12T09:56:37.466268+0000","last_unstale":"2023-12-12T09:57:03.279949+0000","last_undegraded":"2023-12-12T09:57:03.279949+0000","last_fullsized":"2023-12-12T09:57:03.279949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:54:50.868616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.493874+0000","last_change":"2023-12-12T09:56:38.277134+0000","last_active":"2023-12-12T09:56:42.493874+0000","last_peered":"2023-12-12T09:56:42.493874+0000","last_clean":"2023-12-12T09:56:42.493874+0000","last_became_active":"2023-12-12T09:56:38.276701+0000","last_became_peered":"2023-12-12T09:56:38.276701+0000","last_unstale":"2023-12-12T09:56:42.493874+0000","last_undegraded":"2023-12-12T09:56:42.493874+0000","last_fullsized":"2023-12-12T09:56:42.493874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:46:02.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681833+0000","last_change":"2023-12-12T09:56:42.506494+0000","last_active":"2023-12-12T09:56:43.681833+0000","last_peered":"2023-12-12T09:56:43.681833+0000","last_clean":"2023-12-12T09:56:43.681833+0000","last_became_active":"2023-12-12T09:56:42.505965+0000","last_became_peered":"2023-12-12T09:56:42.505965+0000","last_unstale":"2023-12-12T09:56:43.681833+0000","last_undegraded":"2023-12-12T09:56:43.681833+0000","last_fullsized":"2023-12-12T09:56:43.681833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:32:01.294474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.275430+0000","last_change":"2023-12-12T09:56:37.469046+0000","last_active":"2023-12-12T09:57:03.275430+0000","last_peered":"2023-12-12T09:57:03.275430+0000","last_clean":"2023-12-12T09:57:03.275430+0000","last_became_active":"2023-12-12T09:56:37.468112+0000","last_became_peered":"2023-12-12T09:56:37.468112+0000","last_unstale":"2023-12-12T09:57:03.275430+0000","last_undegraded":"2023-12-12T09:57:03.275430+0000","last_fullsized":"2023-12-12T09:57:03.275430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:39:35.356243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496268+0000","last_change":"2023-12-12T09:56:38.280000+0000","last_active":"2023-12-12T09:56:42.496268+0000","last_peered":"2023-12-12T09:56:42.496268+0000","last_clean":"2023-12-12T09:56:42.496268+0000","last_became_active":"2023-12-12T09:56:38.279723+0000","last_became_peered":"2023-12-12T09:56:38.279723+0000","last_unstale":"2023-12-12T09:56:42.496268+0000","last_undegraded":"2023-12-12T09:56:42.496268+0000","last_fullsized":"2023-12-12T09:56:42.496268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:56:35.326614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682388+0000","last_change":"2023-12-12T09:56:42.506820+0000","last_active":"2023-12-12T09:56:43.682388+0000","last_peered":"2023-12-12T09:56:43.682388+0000","last_clean":"2023-12-12T09:56:43.682388+0000","last_became_active":"2023-12-12T09:56:42.506481+0000","last_became_peered":"2023-12-12T09:56:42.506481+0000","last_unstale":"2023-12-12T09:56:43.682388+0000","last_undegraded":"2023-12-12T09:56:43.682388+0000","last_fullsized":"2023-12-12T09:56:43.682388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:10:50.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.299662+0000","last_change":"2023-12-12T09:56:37.470283+0000","last_active":"2023-12-12T09:57:05.299662+0000","last_peered":"2023-12-12T09:57:05.299662+0000","last_clean":"2023-12-12T09:57:05.299662+0000","last_became_active":"2023-12-12T09:56:37.469568+0000","last_became_peered":"2023-12-12T09:56:37.469568+0000","last_unstale":"2023-12-12T09:57:05.299662+0000","last_undegraded":"2023-12-12T09:57:05.299662+0000","last_fullsized":"2023-12-12T09:57:05.299662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:24.427658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.493857+0000","last_change":"2023-12-12T09:56:38.279180+0000","last_active":"2023-12-12T09:56:42.493857+0000","last_peered":"2023-12-12T09:56:42.493857+0000","last_clean":"2023-12-12T09:56:42.493857+0000","last_became_active":"2023-12-12T09:56:38.278493+0000","last_became_peered":"2023-12-12T09:56:38.278493+0000","last_unstale":"2023-12-12T09:56:42.493857+0000","last_undegraded":"2023-12-12T09:56:42.493857+0000","last_fullsized":"2023-12-12T09:56:42.493857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:17:52.468663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682004+0000","last_change":"2023-12-12T09:56:42.505140+0000","last_active":"2023-12-12T09:56:43.682004+0000","last_peered":"2023-12-12T09:56:43.682004+0000","last_clean":"2023-12-12T09:56:43.682004+0000","last_became_active":"2023-12-12T09:56:42.504655+0000","last_became_peered":"2023-12-12T09:56:42.504655+0000","last_unstale":"2023-12-12T09:56:43.682004+0000","last_undegraded":"2023-12-12T09:56:43.682004+0000","last_fullsized":"2023-12-12T09:56:43.682004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:58:18.004084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.279365+0000","last_change":"2023-12-12T09:56:37.468048+0000","last_active":"2023-12-12T09:57:03.279365+0000","last_peered":"2023-12-12T09:57:03.279365+0000","last_clean":"2023-12-12T09:57:03.279365+0000","last_became_active":"2023-12-12T09:56:37.467744+0000","last_became_peered":"2023-12-12T09:56:37.467744+0000","last_unstale":"2023-12-12T09:57:03.279365+0000","last_undegraded":"2023-12-12T09:57:03.279365+0000","last_fullsized":"2023-12-12T09:57:03.279365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:29:56.033230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3840,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.492676+0000","last_change":"2023-12-12T09:56:38.284640+0000","last_active":"2023-12-12T09:56:42.492676+0000","last_peered":"2023-12-12T09:56:42.492676+0000","last_clean":"2023-12-12T09:56:42.492676+0000","last_became_active":"2023-12-12T09:56:38.284351+0000","last_became_peered":"2023-12-12T09:56:38.284351+0000","last_unstale":"2023-12-12T09:56:42.492676+0000","last_undegraded":"2023-12-12T09:56:42.492676+0000","last_fullsized":"2023-12-12T09:56:42.492676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:22.910616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682133+0000","last_change":"2023-12-12T09:56:42.502821+0000","last_active":"2023-12-12T09:56:43.682133+0000","last_peered":"2023-12-12T09:56:43.682133+0000","last_clean":"2023-12-12T09:56:43.682133+0000","last_became_active":"2023-12-12T09:56:42.502377+0000","last_became_peered":"2023-12-12T09:56:42.502377+0000","last_unstale":"2023-12-12T09:56:43.682133+0000","last_undegraded":"2023-12-12T09:56:43.682133+0000","last_fullsized":"2023-12-12T09:56:43.682133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:41:22.913984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.301174+0000","last_change":"2023-12-12T09:56:37.468656+0000","last_active":"2023-12-12T09:57:06.301174+0000","last_peered":"2023-12-12T09:57:06.301174+0000","last_clean":"2023-12-12T09:57:06.301174+0000","last_became_active":"2023-12-12T09:56:37.468353+0000","last_became_peered":"2023-12-12T09:56:37.468353+0000","last_unstale":"2023-12-12T09:57:06.301174+0000","last_undegraded":"2023-12-12T09:57:06.301174+0000","last_fullsized":"2023-12-12T09:57:06.301174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:51:21.477423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.500156+0000","last_change":"2023-12-12T09:56:38.277290+0000","last_active":"2023-12-12T09:56:42.500156+0000","last_peered":"2023-12-12T09:56:42.500156+0000","last_clean":"2023-12-12T09:56:42.500156+0000","last_became_active":"2023-12-12T09:56:38.276972+0000","last_became_peered":"2023-12-12T09:56:38.276972+0000","last_unstale":"2023-12-12T09:56:42.500156+0000","last_undegraded":"2023-12-12T09:56:42.500156+0000","last_fullsized":"2023-12-12T09:56:42.500156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:03:25.254368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680672+0000","last_change":"2023-12-12T09:56:42.506900+0000","last_active":"2023-12-12T09:56:43.680672+0000","last_peered":"2023-12-12T09:56:43.680672+0000","last_clean":"2023-12-12T09:56:43.680672+0000","last_became_active":"2023-12-12T09:56:42.506470+0000","last_became_peered":"2023-12-12T09:56:42.506470+0000","last_unstale":"2023-12-12T09:56:43.680672+0000","last_undegraded":"2023-12-12T09:56:43.680672+0000","last_fullsized":"2023-12-12T09:56:43.680672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:04:17.359766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.299212+0000","last_change":"2023-12-12T09:56:37.472598+0000","last_active":"2023-12-12T09:57:06.299212+0000","last_peered":"2023-12-12T09:57:06.299212+0000","last_clean":"2023-12-12T09:57:06.299212+0000","last_became_active":"2023-12-12T09:56:37.471809+0000","last_became_peered":"2023-12-12T09:56:37.471809+0000","last_unstale":"2023-12-12T09:57:06.299212+0000","last_undegraded":"2023-12-12T09:57:06.299212+0000","last_fullsized":"2023-12-12T09:57:06.299212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:54:08.677081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497069+0000","last_change":"2023-12-12T09:56:38.277180+0000","last_active":"2023-12-12T09:56:42.497069+0000","last_peered":"2023-12-12T09:56:42.497069+0000","last_clean":"2023-12-12T09:56:42.497069+0000","last_became_active":"2023-12-12T09:56:38.276878+0000","last_became_peered":"2023-12-12T09:56:38.276878+0000","last_unstale":"2023-12-12T09:56:42.497069+0000","last_undegraded":"2023-12-12T09:56:42.497069+0000","last_fullsized":"2023-12-12T09:56:42.497069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:18:49.704820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682061+0000","last_change":"2023-12-12T09:56:42.512516+0000","last_active":"2023-12-12T09:56:43.682061+0000","last_peered":"2023-12-12T09:56:43.682061+0000","last_clean":"2023-12-12T09:56:43.682061+0000","last_became_active":"2023-12-12T09:56:42.511682+0000","last_became_peered":"2023-12-12T09:56:42.511682+0000","last_unstale":"2023-12-12T09:56:43.682061+0000","last_undegraded":"2023-12-12T09:56:43.682061+0000","last_fullsized":"2023-12-12T09:56:43.682061+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:34:02.287563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.297091+0000","last_change":"2023-12-12T09:56:37.468329+0000","last_active":"2023-12-12T09:57:05.297091+0000","last_peered":"2023-12-12T09:57:05.297091+0000","last_clean":"2023-12-12T09:57:05.297091+0000","last_became_active":"2023-12-12T09:56:37.467886+0000","last_became_peered":"2023-12-12T09:56:37.467886+0000","last_unstale":"2023-12-12T09:57:05.297091+0000","last_undegraded":"2023-12-12T09:57:05.297091+0000","last_fullsized":"2023-12-12T09:57:05.297091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:23:35.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495836+0000","last_change":"2023-12-12T09:56:38.281280+0000","last_active":"2023-12-12T09:56:42.495836+0000","last_peered":"2023-12-12T09:56:42.495836+0000","last_clean":"2023-12-12T09:56:42.495836+0000","last_became_active":"2023-12-12T09:56:38.280993+0000","last_became_peered":"2023-12-12T09:56:38.280993+0000","last_unstale":"2023-12-12T09:56:42.495836+0000","last_undegraded":"2023-12-12T09:56:42.495836+0000","last_fullsized":"2023-12-12T09:56:42.495836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:53:44.562981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681331+0000","last_change":"2023-12-12T09:56:42.508920+0000","last_active":"2023-12-12T09:56:43.681331+0000","last_peered":"2023-12-12T09:56:43.681331+0000","last_clean":"2023-12-12T09:56:43.681331+0000","last_became_active":"2023-12-12T09:56:42.508491+0000","last_became_peered":"2023-12-12T09:56:42.508491+0000","last_unstale":"2023-12-12T09:56:43.681331+0000","last_undegraded":"2023-12-12T09:56:43.681331+0000","last_fullsized":"2023-12-12T09:56:43.681331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:12:50.596137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.300318+0000","last_change":"2023-12-12T09:56:37.474701+0000","last_active":"2023-12-12T09:57:06.300318+0000","last_peered":"2023-12-12T09:57:06.300318+0000","last_clean":"2023-12-12T09:57:06.300318+0000","last_became_active":"2023-12-12T09:56:37.473093+0000","last_became_peered":"2023-12-12T09:56:37.473093+0000","last_unstale":"2023-12-12T09:57:06.300318+0000","last_undegraded":"2023-12-12T09:57:06.300318+0000","last_fullsized":"2023-12-12T09:57:06.300318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:42:10.274607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.488056+0000","last_change":"2023-12-12T09:56:38.279775+0000","last_active":"2023-12-12T09:56:42.488056+0000","last_peered":"2023-12-12T09:56:42.488056+0000","last_clean":"2023-12-12T09:56:42.488056+0000","last_became_active":"2023-12-12T09:56:38.278612+0000","last_became_peered":"2023-12-12T09:56:38.278612+0000","last_unstale":"2023-12-12T09:56:42.488056+0000","last_undegraded":"2023-12-12T09:56:42.488056+0000","last_fullsized":"2023-12-12T09:56:42.488056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:53:57.915759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681301+0000","last_change":"2023-12-12T09:56:42.499828+0000","last_active":"2023-12-12T09:56:43.681301+0000","last_peered":"2023-12-12T09:56:43.681301+0000","last_clean":"2023-12-12T09:56:43.681301+0000","last_became_active":"2023-12-12T09:56:42.499453+0000","last_became_peered":"2023-12-12T09:56:42.499453+0000","last_unstale":"2023-12-12T09:56:43.681301+0000","last_undegraded":"2023-12-12T09:56:43.681301+0000","last_fullsized":"2023-12-12T09:56:43.681301+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:20:12.609145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.785179+0000","last_change":"2023-12-12T09:56:37.475428+0000","last_active":"2023-12-12T09:57:05.785179+0000","last_peered":"2023-12-12T09:57:05.785179+0000","last_clean":"2023-12-12T09:57:05.785179+0000","last_became_active":"2023-12-12T09:56:37.473070+0000","last_became_peered":"2023-12-12T09:56:37.473070+0000","last_unstale":"2023-12-12T09:57:05.785179+0000","last_undegraded":"2023-12-12T09:57:05.785179+0000","last_fullsized":"2023-12-12T09:57:05.785179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:04:52.292339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.492247+0000","last_change":"2023-12-12T09:56:38.281060+0000","last_active":"2023-12-12T09:56:42.492247+0000","last_peered":"2023-12-12T09:56:42.492247+0000","last_clean":"2023-12-12T09:56:42.492247+0000","last_became_active":"2023-12-12T09:56:38.280725+0000","last_became_peered":"2023-12-12T09:56:38.280725+0000","last_unstale":"2023-12-12T09:56:42.492247+0000","last_undegraded":"2023-12-12T09:56:42.492247+0000","last_fullsized":"2023-12-12T09:56:42.492247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:14.592213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.262731+0000","last_change":"2023-12-12T09:56:42.578104+0000","last_active":"2023-12-12T09:56:43.262731+0000","last_peered":"2023-12-12T09:56:43.262731+0000","last_clean":"2023-12-12T09:56:43.262731+0000","last_became_active":"2023-12-12T09:56:42.577598+0000","last_became_peered":"2023-12-12T09:56:42.577598+0000","last_unstale":"2023-12-12T09:56:43.262731+0000","last_undegraded":"2023-12-12T09:56:43.262731+0000","last_fullsized":"2023-12-12T09:56:43.262731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:27:49.590817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.500272+0000","last_change":"2023-12-12T09:56:37.465071+0000","last_active":"2023-12-12T09:56:42.500272+0000","last_peered":"2023-12-12T09:56:42.500272+0000","last_clean":"2023-12-12T09:56:42.500272+0000","last_became_active":"2023-12-12T09:56:37.464717+0000","last_became_peered":"2023-12-12T09:56:37.464717+0000","last_unstale":"2023-12-12T09:56:42.500272+0000","last_undegraded":"2023-12-12T09:56:42.500272+0000","last_fullsized":"2023-12-12T09:56:42.500272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:04:52.167228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576905+0000","last_change":"2023-12-12T09:56:38.273276+0000","last_active":"2023-12-12T09:56:42.576905+0000","last_peered":"2023-12-12T09:56:42.576905+0000","last_clean":"2023-12-12T09:56:42.576905+0000","last_became_active":"2023-12-12T09:56:38.272966+0000","last_became_peered":"2023-12-12T09:56:38.272966+0000","last_unstale":"2023-12-12T09:56:42.576905+0000","last_undegraded":"2023-12-12T09:56:42.576905+0000","last_fullsized":"2023-12-12T09:56:42.576905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:22.352843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680378+0000","last_change":"2023-12-12T09:56:42.510808+0000","last_active":"2023-12-12T09:56:43.680378+0000","last_peered":"2023-12-12T09:56:43.680378+0000","last_clean":"2023-12-12T09:56:43.680378+0000","last_became_active":"2023-12-12T09:56:42.510408+0000","last_became_peered":"2023-12-12T09:56:42.510408+0000","last_unstale":"2023-12-12T09:56:43.680378+0000","last_undegraded":"2023-12-12T09:56:43.680378+0000","last_fullsized":"2023-12-12T09:56:43.680378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:53:08.573345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.263065+0000","last_change":"2023-12-12T09:56:37.468863+0000","last_active":"2023-12-12T09:56:43.263065+0000","last_peered":"2023-12-12T09:56:43.263065+0000","last_clean":"2023-12-12T09:56:43.263065+0000","last_became_active":"2023-12-12T09:56:37.468221+0000","last_became_peered":"2023-12-12T09:56:37.468221+0000","last_unstale":"2023-12-12T09:56:43.263065+0000","last_undegraded":"2023-12-12T09:56:43.263065+0000","last_fullsized":"2023-12-12T09:56:43.263065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:39:17.559717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576920+0000","last_change":"2023-12-12T09:56:38.277983+0000","last_active":"2023-12-12T09:56:42.576920+0000","last_peered":"2023-12-12T09:56:42.576920+0000","last_clean":"2023-12-12T09:56:42.576920+0000","last_became_active":"2023-12-12T09:56:38.275926+0000","last_became_peered":"2023-12-12T09:56:38.275926+0000","last_unstale":"2023-12-12T09:56:42.576920+0000","last_undegraded":"2023-12-12T09:56:42.576920+0000","last_fullsized":"2023-12-12T09:56:42.576920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:57:51.474651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680842+0000","last_change":"2023-12-12T09:56:42.577662+0000","last_active":"2023-12-12T09:56:43.680842+0000","last_peered":"2023-12-12T09:56:43.680842+0000","last_clean":"2023-12-12T09:56:43.680842+0000","last_became_active":"2023-12-12T09:56:42.577188+0000","last_became_peered":"2023-12-12T09:56:42.577188+0000","last_unstale":"2023-12-12T09:56:43.680842+0000","last_undegraded":"2023-12-12T09:56:43.680842+0000","last_fullsized":"2023-12-12T09:56:43.680842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:58:35.341894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487726+0000","last_change":"2023-12-12T09:56:38.279537+0000","last_active":"2023-12-12T09:56:42.487726+0000","last_peered":"2023-12-12T09:56:42.487726+0000","last_clean":"2023-12-12T09:56:42.487726+0000","last_became_active":"2023-12-12T09:56:38.278739+0000","last_became_peered":"2023-12-12T09:56:38.278739+0000","last_unstale":"2023-12-12T09:56:42.487726+0000","last_undegraded":"2023-12-12T09:56:42.487726+0000","last_fullsized":"2023-12-12T09:56:42.487726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:35:05.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680095+0000","last_change":"2023-12-12T09:56:37.465726+0000","last_active":"2023-12-12T09:56:43.680095+0000","last_peered":"2023-12-12T09:56:43.680095+0000","last_clean":"2023-12-12T09:56:43.680095+0000","last_became_active":"2023-12-12T09:56:37.465438+0000","last_became_peered":"2023-12-12T09:56:37.465438+0000","last_unstale":"2023-12-12T09:56:43.680095+0000","last_undegraded":"2023-12-12T09:56:43.680095+0000","last_fullsized":"2023-12-12T09:56:43.680095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:17:58.356893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.263337+0000","last_change":"2023-12-12T09:56:42.505943+0000","last_active":"2023-12-12T09:56:43.263337+0000","last_peered":"2023-12-12T09:56:43.263337+0000","last_clean":"2023-12-12T09:56:43.263337+0000","last_became_active":"2023-12-12T09:56:42.505581+0000","last_became_peered":"2023-12-12T09:56:42.505581+0000","last_unstale":"2023-12-12T09:56:43.263337+0000","last_undegraded":"2023-12-12T09:56:43.263337+0000","last_fullsized":"2023-12-12T09:56:43.263337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:19:08.869868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.488597+0000","last_change":"2023-12-12T09:56:38.284895+0000","last_active":"2023-12-12T09:56:42.488597+0000","last_peered":"2023-12-12T09:56:42.488597+0000","last_clean":"2023-12-12T09:56:42.488597+0000","last_became_active":"2023-12-12T09:56:38.284429+0000","last_became_peered":"2023-12-12T09:56:38.284429+0000","last_unstale":"2023-12-12T09:56:42.488597+0000","last_undegraded":"2023-12-12T09:56:42.488597+0000","last_fullsized":"2023-12-12T09:56:42.488597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:14.170629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.300517+0000","last_change":"2023-12-12T09:56:37.466748+0000","last_active":"2023-12-12T09:57:05.300517+0000","last_peered":"2023-12-12T09:57:05.300517+0000","last_clean":"2023-12-12T09:57:05.300517+0000","last_became_active":"2023-12-12T09:56:37.466459+0000","last_became_peered":"2023-12-12T09:56:37.466459+0000","last_unstale":"2023-12-12T09:57:05.300517+0000","last_undegraded":"2023-12-12T09:57:05.300517+0000","last_fullsized":"2023-12-12T09:57:05.300517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:21:43.819845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495646+0000","last_change":"2023-12-12T09:55:37.275186+0000","last_active":"2023-12-12T09:56:42.495646+0000","last_peered":"2023-12-12T09:56:42.495646+0000","last_clean":"2023-12-12T09:56:42.495646+0000","last_became_active":"2023-12-12T09:55:37.274808+0000","last_became_peered":"2023-12-12T09:55:37.274808+0000","last_unstale":"2023-12-12T09:56:42.495646+0000","last_undegraded":"2023-12-12T09:56:42.495646+0000","last_fullsized":"2023-12-12T09:56:42.495646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:42.502190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:42.502190+0000","last_clean_scrub_stamp":"2023-12-12T09:52:42.502190+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:27:08.942642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681726+0000","last_change":"2023-12-12T09:56:42.578219+0000","last_active":"2023-12-12T09:56:43.681726+0000","last_peered":"2023-12-12T09:56:43.681726+0000","last_clean":"2023-12-12T09:56:43.681726+0000","last_became_active":"2023-12-12T09:56:42.577589+0000","last_became_peered":"2023-12-12T09:56:42.577589+0000","last_unstale":"2023-12-12T09:56:43.681726+0000","last_undegraded":"2023-12-12T09:56:43.681726+0000","last_fullsized":"2023-12-12T09:56:43.681726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:40:54.162235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499183+0000","last_change":"2023-12-12T09:56:38.274163+0000","last_active":"2023-12-12T09:56:42.499183+0000","last_peered":"2023-12-12T09:56:42.499183+0000","last_clean":"2023-12-12T09:56:42.499183+0000","last_became_active":"2023-12-12T09:56:38.273837+0000","last_became_peered":"2023-12-12T09:56:38.273837+0000","last_unstale":"2023-12-12T09:56:42.499183+0000","last_undegraded":"2023-12-12T09:56:42.499183+0000","last_fullsized":"2023-12-12T09:56:42.499183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:46:53.550289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:04.286740+0000","last_change":"2023-12-12T09:56:37.473275+0000","last_active":"2023-12-12T09:57:04.286740+0000","last_peered":"2023-12-12T09:57:04.286740+0000","last_clean":"2023-12-12T09:57:04.286740+0000","last_became_active":"2023-12-12T09:56:37.472973+0000","last_became_peered":"2023-12-12T09:56:37.472973+0000","last_unstale":"2023-12-12T09:57:04.286740+0000","last_undegraded":"2023-12-12T09:57:04.286740+0000","last_fullsized":"2023-12-12T09:57:04.286740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:03.660259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680813+0000","last_change":"2023-12-12T09:56:42.503299+0000","last_active":"2023-12-12T09:56:43.680813+0000","last_peered":"2023-12-12T09:56:43.680813+0000","last_clean":"2023-12-12T09:56:43.680813+0000","last_became_active":"2023-12-12T09:56:42.502862+0000","last_became_peered":"2023-12-12T09:56:42.502862+0000","last_unstale":"2023-12-12T09:56:43.680813+0000","last_undegraded":"2023-12-12T09:56:43.680813+0000","last_fullsized":"2023-12-12T09:56:43.680813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:47:14.211916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.301281+0000","last_change":"2023-12-12T09:56:37.476084+0000","last_active":"2023-12-12T09:57:06.301281+0000","last_peered":"2023-12-12T09:57:06.301281+0000","last_clean":"2023-12-12T09:57:06.301281+0000","last_became_active":"2023-12-12T09:56:37.475760+0000","last_became_peered":"2023-12-12T09:56:37.475760+0000","last_unstale":"2023-12-12T09:57:06.301281+0000","last_undegraded":"2023-12-12T09:57:06.301281+0000","last_fullsized":"2023-12-12T09:57:06.301281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:30:43.487809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499907+0000","last_change":"2023-12-12T09:56:38.278502+0000","last_active":"2023-12-12T09:56:42.499907+0000","last_peered":"2023-12-12T09:56:42.499907+0000","last_clean":"2023-12-12T09:56:42.499907+0000","last_became_active":"2023-12-12T09:56:38.278210+0000","last_became_peered":"2023-12-12T09:56:38.278210+0000","last_unstale":"2023-12-12T09:56:42.499907+0000","last_undegraded":"2023-12-12T09:56:42.499907+0000","last_fullsized":"2023-12-12T09:56:42.499907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:25:09.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680508+0000","last_change":"2023-12-12T09:56:42.577598+0000","last_active":"2023-12-12T09:56:43.680508+0000","last_peered":"2023-12-12T09:56:43.680508+0000","last_clean":"2023-12-12T09:56:43.680508+0000","last_became_active":"2023-12-12T09:56:42.577147+0000","last_became_peered":"2023-12-12T09:56:42.577147+0000","last_unstale":"2023-12-12T09:56:43.680508+0000","last_undegraded":"2023-12-12T09:56:43.680508+0000","last_fullsized":"2023-12-12T09:56:43.680508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:05:11.279871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:04.287878+0000","last_change":"2023-12-12T09:56:37.474862+0000","last_active":"2023-12-12T09:57:04.287878+0000","last_peered":"2023-12-12T09:57:04.287878+0000","last_clean":"2023-12-12T09:57:04.287878+0000","last_became_active":"2023-12-12T09:56:37.474463+0000","last_became_peered":"2023-12-12T09:56:37.474463+0000","last_unstale":"2023-12-12T09:57:04.287878+0000","last_undegraded":"2023-12-12T09:57:04.287878+0000","last_fullsized":"2023-12-12T09:57:04.287878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:06.926221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487901+0000","last_change":"2023-12-12T09:56:38.279729+0000","last_active":"2023-12-12T09:56:42.487901+0000","last_peered":"2023-12-12T09:56:42.487901+0000","last_clean":"2023-12-12T09:56:42.487901+0000","last_became_active":"2023-12-12T09:56:38.277634+0000","last_became_peered":"2023-12-12T09:56:38.277634+0000","last_unstale":"2023-12-12T09:56:42.487901+0000","last_undegraded":"2023-12-12T09:56:42.487901+0000","last_fullsized":"2023-12-12T09:56:42.487901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:06:22.594507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.554701+0000","last_change":"2023-12-12T09:56:42.502342+0000","last_active":"2023-12-12T09:56:43.554701+0000","last_peered":"2023-12-12T09:56:43.554701+0000","last_clean":"2023-12-12T09:56:43.554701+0000","last_became_active":"2023-12-12T09:56:42.501313+0000","last_became_peered":"2023-12-12T09:56:42.501313+0000","last_unstale":"2023-12-12T09:56:43.554701+0000","last_undegraded":"2023-12-12T09:56:43.554701+0000","last_fullsized":"2023-12-12T09:56:43.554701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:12:28.144029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.299191+0000","last_change":"2023-12-12T09:56:37.464428+0000","last_active":"2023-12-12T09:57:06.299191+0000","last_peered":"2023-12-12T09:57:06.299191+0000","last_clean":"2023-12-12T09:57:06.299191+0000","last_became_active":"2023-12-12T09:56:37.464159+0000","last_became_peered":"2023-12-12T09:56:37.464159+0000","last_unstale":"2023-12-12T09:57:06.299191+0000","last_undegraded":"2023-12-12T09:57:06.299191+0000","last_fullsized":"2023-12-12T09:57:06.299191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:14:53.779852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496677+0000","last_change":"2023-12-12T09:56:38.277023+0000","last_active":"2023-12-12T09:56:42.496677+0000","last_peered":"2023-12-12T09:56:42.496677+0000","last_clean":"2023-12-12T09:56:42.496677+0000","last_became_active":"2023-12-12T09:56:38.276708+0000","last_became_peered":"2023-12-12T09:56:38.276708+0000","last_unstale":"2023-12-12T09:56:42.496677+0000","last_undegraded":"2023-12-12T09:56:42.496677+0000","last_fullsized":"2023-12-12T09:56:42.496677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:08:28.244491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681673+0000","last_change":"2023-12-12T09:56:42.578025+0000","last_active":"2023-12-12T09:56:43.681673+0000","last_peered":"2023-12-12T09:56:43.681673+0000","last_clean":"2023-12-12T09:56:43.681673+0000","last_became_active":"2023-12-12T09:56:42.577570+0000","last_became_peered":"2023-12-12T09:56:42.577570+0000","last_unstale":"2023-12-12T09:56:43.681673+0000","last_undegraded":"2023-12-12T09:56:43.681673+0000","last_fullsized":"2023-12-12T09:56:43.681673+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:51:02.998355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.276026+0000","last_change":"2023-12-12T09:56:37.467815+0000","last_active":"2023-12-12T09:57:03.276026+0000","last_peered":"2023-12-12T09:57:03.276026+0000","last_clean":"2023-12-12T09:57:03.276026+0000","last_became_active":"2023-12-12T09:56:37.467110+0000","last_became_peered":"2023-12-12T09:56:37.467110+0000","last_unstale":"2023-12-12T09:57:03.276026+0000","last_undegraded":"2023-12-12T09:57:03.276026+0000","last_fullsized":"2023-12-12T09:57:03.276026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:40:24.325774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.491987+0000","last_change":"2023-12-12T09:56:38.284770+0000","last_active":"2023-12-12T09:56:42.491987+0000","last_peered":"2023-12-12T09:56:42.491987+0000","last_clean":"2023-12-12T09:56:42.491987+0000","last_became_active":"2023-12-12T09:56:38.284459+0000","last_became_peered":"2023-12-12T09:56:38.284459+0000","last_unstale":"2023-12-12T09:56:42.491987+0000","last_undegraded":"2023-12-12T09:56:42.491987+0000","last_fullsized":"2023-12-12T09:56:42.491987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:02:42.787163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682121+0000","last_change":"2023-12-12T09:56:42.512278+0000","last_active":"2023-12-12T09:56:43.682121+0000","last_peered":"2023-12-12T09:56:43.682121+0000","last_clean":"2023-12-12T09:56:43.682121+0000","last_became_active":"2023-12-12T09:56:42.511598+0000","last_became_peered":"2023-12-12T09:56:42.511598+0000","last_unstale":"2023-12-12T09:56:43.682121+0000","last_undegraded":"2023-12-12T09:56:43.682121+0000","last_fullsized":"2023-12-12T09:56:43.682121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:00:03.571614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.298234+0000","last_change":"2023-12-12T09:56:37.469274+0000","last_active":"2023-12-12T09:57:05.298234+0000","last_peered":"2023-12-12T09:57:05.298234+0000","last_clean":"2023-12-12T09:57:05.298234+0000","last_became_active":"2023-12-12T09:56:37.468947+0000","last_became_peered":"2023-12-12T09:56:37.468947+0000","last_unstale":"2023-12-12T09:57:05.298234+0000","last_undegraded":"2023-12-12T09:57:05.298234+0000","last_fullsized":"2023-12-12T09:57:05.298234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:26:25.559385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.491775+0000","last_change":"2023-12-12T09:56:38.274415+0000","last_active":"2023-12-12T09:56:42.491775+0000","last_peered":"2023-12-12T09:56:42.491775+0000","last_clean":"2023-12-12T09:56:42.491775+0000","last_became_active":"2023-12-12T09:56:38.274119+0000","last_became_peered":"2023-12-12T09:56:38.274119+0000","last_unstale":"2023-12-12T09:56:42.491775+0000","last_undegraded":"2023-12-12T09:56:42.491775+0000","last_fullsized":"2023-12-12T09:56:42.491775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:50.160355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681718+0000","last_change":"2023-12-12T09:56:42.506811+0000","last_active":"2023-12-12T09:56:43.681718+0000","last_peered":"2023-12-12T09:56:43.681718+0000","last_clean":"2023-12-12T09:56:43.681718+0000","last_became_active":"2023-12-12T09:56:42.506374+0000","last_became_peered":"2023-12-12T09:56:42.506374+0000","last_unstale":"2023-12-12T09:56:43.681718+0000","last_undegraded":"2023-12-12T09:56:43.681718+0000","last_fullsized":"2023-12-12T09:56:43.681718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:50:54.647403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.218120+0000","last_change":"2023-12-12T09:56:37.473548+0000","last_active":"2023-12-12T09:57:05.218120+0000","last_peered":"2023-12-12T09:57:05.218120+0000","last_clean":"2023-12-12T09:57:05.218120+0000","last_became_active":"2023-12-12T09:56:37.473240+0000","last_became_peered":"2023-12-12T09:56:37.473240+0000","last_unstale":"2023-12-12T09:57:05.218120+0000","last_undegraded":"2023-12-12T09:57:05.218120+0000","last_fullsized":"2023-12-12T09:57:05.218120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:42:22.921437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487444+0000","last_change":"2023-12-12T09:56:38.276853+0000","last_active":"2023-12-12T09:56:42.487444+0000","last_peered":"2023-12-12T09:56:42.487444+0000","last_clean":"2023-12-12T09:56:42.487444+0000","last_became_active":"2023-12-12T09:56:38.276567+0000","last_became_peered":"2023-12-12T09:56:38.276567+0000","last_unstale":"2023-12-12T09:56:42.487444+0000","last_undegraded":"2023-12-12T09:56:42.487444+0000","last_fullsized":"2023-12-12T09:56:42.487444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:21:15.069389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681616+0000","last_change":"2023-12-12T09:56:42.579135+0000","last_active":"2023-12-12T09:56:43.681616+0000","last_peered":"2023-12-12T09:56:43.681616+0000","last_clean":"2023-12-12T09:56:43.681616+0000","last_became_active":"2023-12-12T09:56:42.578513+0000","last_became_peered":"2023-12-12T09:56:42.578513+0000","last_unstale":"2023-12-12T09:56:43.681616+0000","last_undegraded":"2023-12-12T09:56:43.681616+0000","last_fullsized":"2023-12-12T09:56:43.681616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:36:27.337524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.276148+0000","last_change":"2023-12-12T09:56:37.475479+0000","last_active":"2023-12-12T09:57:03.276148+0000","last_peered":"2023-12-12T09:57:03.276148+0000","last_clean":"2023-12-12T09:57:03.276148+0000","last_became_active":"2023-12-12T09:56:37.473657+0000","last_became_peered":"2023-12-12T09:56:37.473657+0000","last_unstale":"2023-12-12T09:57:03.276148+0000","last_undegraded":"2023-12-12T09:57:03.276148+0000","last_fullsized":"2023-12-12T09:57:03.276148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:49:07.610160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496445+0000","last_change":"2023-12-12T09:56:38.279661+0000","last_active":"2023-12-12T09:56:42.496445+0000","last_peered":"2023-12-12T09:56:42.496445+0000","last_clean":"2023-12-12T09:56:42.496445+0000","last_became_active":"2023-12-12T09:56:38.279330+0000","last_became_peered":"2023-12-12T09:56:38.279330+0000","last_unstale":"2023-12-12T09:56:42.496445+0000","last_undegraded":"2023-12-12T09:56:42.496445+0000","last_fullsized":"2023-12-12T09:56:42.496445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:19:15.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680487+0000","last_change":"2023-12-12T09:56:42.502911+0000","last_active":"2023-12-12T09:56:43.680487+0000","last_peered":"2023-12-12T09:56:43.680487+0000","last_clean":"2023-12-12T09:56:43.680487+0000","last_became_active":"2023-12-12T09:56:42.502467+0000","last_became_peered":"2023-12-12T09:56:42.502467+0000","last_unstale":"2023-12-12T09:56:43.680487+0000","last_undegraded":"2023-12-12T09:56:43.680487+0000","last_fullsized":"2023-12-12T09:56:43.680487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:45:42.568227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.296154+0000","last_change":"2023-12-12T09:56:37.466958+0000","last_active":"2023-12-12T09:57:05.296154+0000","last_peered":"2023-12-12T09:57:05.296154+0000","last_clean":"2023-12-12T09:57:05.296154+0000","last_became_active":"2023-12-12T09:56:37.466591+0000","last_became_peered":"2023-12-12T09:56:37.466591+0000","last_unstale":"2023-12-12T09:57:05.296154+0000","last_undegraded":"2023-12-12T09:57:05.296154+0000","last_fullsized":"2023-12-12T09:57:05.296154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:25:35.605587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497525+0000","last_change":"2023-12-12T09:56:38.279142+0000","last_active":"2023-12-12T09:56:42.497525+0000","last_peered":"2023-12-12T09:56:42.497525+0000","last_clean":"2023-12-12T09:56:42.497525+0000","last_became_active":"2023-12-12T09:56:38.278441+0000","last_became_peered":"2023-12-12T09:56:38.278441+0000","last_unstale":"2023-12-12T09:56:42.497525+0000","last_undegraded":"2023-12-12T09:56:42.497525+0000","last_fullsized":"2023-12-12T09:56:42.497525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:36:49.024628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.261911+0000","last_change":"2023-12-12T09:56:42.506159+0000","last_active":"2023-12-12T09:56:43.261911+0000","last_peered":"2023-12-12T09:56:43.261911+0000","last_clean":"2023-12-12T09:56:43.261911+0000","last_became_active":"2023-12-12T09:56:42.505704+0000","last_became_peered":"2023-12-12T09:56:42.505704+0000","last_unstale":"2023-12-12T09:56:43.261911+0000","last_undegraded":"2023-12-12T09:56:43.261911+0000","last_fullsized":"2023-12-12T09:56:43.261911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:51:55.633237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.300168+0000","last_change":"2023-12-12T09:56:37.470470+0000","last_active":"2023-12-12T09:57:06.300168+0000","last_peered":"2023-12-12T09:57:06.300168+0000","last_clean":"2023-12-12T09:57:06.300168+0000","last_became_active":"2023-12-12T09:56:37.469991+0000","last_became_peered":"2023-12-12T09:56:37.469991+0000","last_unstale":"2023-12-12T09:57:06.300168+0000","last_undegraded":"2023-12-12T09:57:06.300168+0000","last_fullsized":"2023-12-12T09:57:06.300168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:49.239075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496992+0000","last_change":"2023-12-12T09:56:38.281338+0000","last_active":"2023-12-12T09:56:42.496992+0000","last_peered":"2023-12-12T09:56:42.496992+0000","last_clean":"2023-12-12T09:56:42.496992+0000","last_became_active":"2023-12-12T09:56:38.281048+0000","last_became_peered":"2023-12-12T09:56:38.281048+0000","last_unstale":"2023-12-12T09:56:42.496992+0000","last_undegraded":"2023-12-12T09:56:42.496992+0000","last_fullsized":"2023-12-12T09:56:42.496992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:27:27.940574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680622+0000","last_change":"2023-12-12T09:56:42.500612+0000","last_active":"2023-12-12T09:56:43.680622+0000","last_peered":"2023-12-12T09:56:43.680622+0000","last_clean":"2023-12-12T09:56:43.680622+0000","last_became_active":"2023-12-12T09:56:42.500202+0000","last_became_peered":"2023-12-12T09:56:42.500202+0000","last_unstale":"2023-12-12T09:56:43.680622+0000","last_undegraded":"2023-12-12T09:56:43.680622+0000","last_fullsized":"2023-12-12T09:56:43.680622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:11:07.397950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499211+0000","last_change":"2023-12-12T09:56:37.465748+0000","last_active":"2023-12-12T09:56:42.499211+0000","last_peered":"2023-12-12T09:56:42.499211+0000","last_clean":"2023-12-12T09:56:42.499211+0000","last_became_active":"2023-12-12T09:56:37.465408+0000","last_became_peered":"2023-12-12T09:56:37.465408+0000","last_unstale":"2023-12-12T09:56:42.499211+0000","last_undegraded":"2023-12-12T09:56:42.499211+0000","last_fullsized":"2023-12-12T09:56:42.499211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:58:38.777008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497525+0000","last_change":"2023-12-12T09:56:38.283934+0000","last_active":"2023-12-12T09:56:42.497525+0000","last_peered":"2023-12-12T09:56:42.497525+0000","last_clean":"2023-12-12T09:56:42.497525+0000","last_became_active":"2023-12-12T09:56:38.283605+0000","last_became_peered":"2023-12-12T09:56:38.283605+0000","last_unstale":"2023-12-12T09:56:42.497525+0000","last_undegraded":"2023-12-12T09:56:42.497525+0000","last_fullsized":"2023-12-12T09:56:42.497525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:52:46.792159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.491862+0000","last_change":"2023-12-12T09:56:38.279834+0000","last_active":"2023-12-12T09:56:42.491862+0000","last_peered":"2023-12-12T09:56:42.491862+0000","last_clean":"2023-12-12T09:56:42.491862+0000","last_became_active":"2023-12-12T09:56:38.279353+0000","last_became_peered":"2023-12-12T09:56:38.279353+0000","last_unstale":"2023-12-12T09:56:42.491862+0000","last_undegraded":"2023-12-12T09:56:42.491862+0000","last_fullsized":"2023-12-12T09:56:42.491862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:21:52.237458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497845+0000","last_change":"2023-12-12T09:56:38.279341+0000","last_active":"2023-12-12T09:56:42.497845+0000","last_peered":"2023-12-12T09:56:42.497845+0000","last_clean":"2023-12-12T09:56:42.497845+0000","last_became_active":"2023-12-12T09:56:38.279054+0000","last_became_peered":"2023-12-12T09:56:38.279054+0000","last_unstale":"2023-12-12T09:56:42.497845+0000","last_undegraded":"2023-12-12T09:56:42.497845+0000","last_fullsized":"2023-12-12T09:56:42.497845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:55:33.729689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496023+0000","last_change":"2023-12-12T09:56:38.277052+0000","last_active":"2023-12-12T09:56:42.496023+0000","last_peered":"2023-12-12T09:56:42.496023+0000","last_clean":"2023-12-12T09:56:42.496023+0000","last_became_active":"2023-12-12T09:56:38.276236+0000","last_became_peered":"2023-12-12T09:56:38.276236+0000","last_unstale":"2023-12-12T09:56:42.496023+0000","last_undegraded":"2023-12-12T09:56:42.496023+0000","last_fullsized":"2023-12-12T09:56:42.496023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:31:52.066130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494775+0000","last_change":"2023-12-12T09:56:38.280936+0000","last_active":"2023-12-12T09:56:42.494775+0000","last_peered":"2023-12-12T09:56:42.494775+0000","last_clean":"2023-12-12T09:56:42.494775+0000","last_became_active":"2023-12-12T09:56:38.280600+0000","last_became_peered":"2023-12-12T09:56:38.280600+0000","last_unstale":"2023-12-12T09:56:42.494775+0000","last_undegraded":"2023-12-12T09:56:42.494775+0000","last_fullsized":"2023-12-12T09:56:42.494775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:40:36.636954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499777+0000","last_change":"2023-12-12T09:56:38.278839+0000","last_active":"2023-12-12T09:56:42.499777+0000","last_peered":"2023-12-12T09:56:42.499777+0000","last_clean":"2023-12-12T09:56:42.499777+0000","last_became_active":"2023-12-12T09:56:38.276299+0000","last_became_peered":"2023-12-12T09:56:38.276299+0000","last_unstale":"2023-12-12T09:56:42.499777+0000","last_undegraded":"2023-12-12T09:56:42.499777+0000","last_fullsized":"2023-12-12T09:56:42.499777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:38:26.830037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.492126+0000","last_change":"2023-12-12T09:56:38.282200+0000","last_active":"2023-12-12T09:56:42.492126+0000","last_peered":"2023-12-12T09:56:42.492126+0000","last_clean":"2023-12-12T09:56:42.492126+0000","last_became_active":"2023-12-12T09:56:38.281938+0000","last_became_peered":"2023-12-12T09:56:38.281938+0000","last_unstale":"2023-12-12T09:56:42.492126+0000","last_undegraded":"2023-12-12T09:56:42.492126+0000","last_fullsized":"2023-12-12T09:56:42.492126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:45:14.391963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496626+0000","last_change":"2023-12-12T09:56:38.276857+0000","last_active":"2023-12-12T09:56:42.496626+0000","last_peered":"2023-12-12T09:56:42.496626+0000","last_clean":"2023-12-12T09:56:42.496626+0000","last_became_active":"2023-12-12T09:56:38.276562+0000","last_became_peered":"2023-12-12T09:56:38.276562+0000","last_unstale":"2023-12-12T09:56:42.496626+0000","last_undegraded":"2023-12-12T09:56:42.496626+0000","last_fullsized":"2023-12-12T09:56:42.496626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:54:28.262613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.488506+0000","last_change":"2023-12-12T09:56:38.278006+0000","last_active":"2023-12-12T09:56:42.488506+0000","last_peered":"2023-12-12T09:56:42.488506+0000","last_clean":"2023-12-12T09:56:42.488506+0000","last_became_active":"2023-12-12T09:56:38.277222+0000","last_became_peered":"2023-12-12T09:56:38.277222+0000","last_unstale":"2023-12-12T09:56:42.488506+0000","last_undegraded":"2023-12-12T09:56:42.488506+0000","last_fullsized":"2023-12-12T09:56:42.488506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:46:50.602603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487631+0000","last_change":"2023-12-12T09:56:38.282872+0000","last_active":"2023-12-12T09:56:42.487631+0000","last_peered":"2023-12-12T09:56:42.487631+0000","last_clean":"2023-12-12T09:56:42.487631+0000","last_became_active":"2023-12-12T09:56:38.282582+0000","last_became_peered":"2023-12-12T09:56:38.282582+0000","last_unstale":"2023-12-12T09:56:42.487631+0000","last_undegraded":"2023-12-12T09:56:42.487631+0000","last_fullsized":"2023-12-12T09:56:42.487631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:36:35.447323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.486961+0000","last_change":"2023-12-12T09:56:38.286707+0000","last_active":"2023-12-12T09:56:42.486961+0000","last_peered":"2023-12-12T09:56:42.486961+0000","last_clean":"2023-12-12T09:56:42.486961+0000","last_became_active":"2023-12-12T09:56:38.286382+0000","last_became_peered":"2023-12-12T09:56:38.286382+0000","last_unstale":"2023-12-12T09:56:42.486961+0000","last_undegraded":"2023-12-12T09:56:42.486961+0000","last_fullsized":"2023-12-12T09:56:42.486961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:10:12.412091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496230+0000","last_change":"2023-12-12T09:56:38.276461+0000","last_active":"2023-12-12T09:56:42.496230+0000","last_peered":"2023-12-12T09:56:42.496230+0000","last_clean":"2023-12-12T09:56:42.496230+0000","last_became_active":"2023-12-12T09:56:38.276173+0000","last_became_peered":"2023-12-12T09:56:38.276173+0000","last_unstale":"2023-12-12T09:56:42.496230+0000","last_undegraded":"2023-12-12T09:56:42.496230+0000","last_fullsized":"2023-12-12T09:56:42.496230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:03:53.874132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499853+0000","last_change":"2023-12-12T09:56:38.276993+0000","last_active":"2023-12-12T09:56:42.499853+0000","last_peered":"2023-12-12T09:56:42.499853+0000","last_clean":"2023-12-12T09:56:42.499853+0000","last_became_active":"2023-12-12T09:56:38.276629+0000","last_became_peered":"2023-12-12T09:56:38.276629+0000","last_unstale":"2023-12-12T09:56:42.499853+0000","last_undegraded":"2023-12-12T09:56:42.499853+0000","last_fullsized":"2023-12-12T09:56:42.499853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:17:39.363638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.493210+0000","last_change":"2023-12-12T09:56:38.281874+0000","last_active":"2023-12-12T09:56:42.493210+0000","last_peered":"2023-12-12T09:56:42.493210+0000","last_clean":"2023-12-12T09:56:42.493210+0000","last_became_active":"2023-12-12T09:56:38.281563+0000","last_became_peered":"2023-12-12T09:56:38.281563+0000","last_unstale":"2023-12-12T09:56:42.493210+0000","last_undegraded":"2023-12-12T09:56:42.493210+0000","last_fullsized":"2023-12-12T09:56:42.493210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:32:18.886026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10158,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":163,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":356352,"data_stored":72011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449104,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":467691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.57799999999999996}]},{"osd":1,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.75},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.88200000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.97799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":9,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90500000000000003}]},{"osd":10,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.92600000000000005}]}]},{"osd":10,"up_from":70,"seq":300647710740,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1061942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.71299999999999997}]},{"osd":2,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.753},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72699999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68500000000000005}]},{"osd":4,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53400000000000003}]},{"osd":5,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.78600000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.628},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.80700000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.48199999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.878}]},{"osd":9,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.81999999999999995}]},{"osd":11,"last update":"Tue Dec 12 09:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.84399999999999997}]}]},{"osd":9,"up_from":65,"seq":279172874265,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":467927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53300000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.437}]},{"osd":3,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.621},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.504}]},{"osd":6,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.179},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.66000000000000003}]},{"osd":8,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.49299999999999999}]},{"osd":10,"last update":"Tue Dec 12 09:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.73099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135902,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":472705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.44600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.78800000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.436},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.307,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59399999999999997}]},{"osd":3,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.74399999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.752,"5min":0.874,"15min":0.874},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70499999999999996},"min":{"1min":0.53700000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.73199999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95499999999999996,"5min":1.117,"15min":1.117},"last":0.73099999999999998}]},{"osd":9,"last update":"Tue Dec 12 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70899999999999996}]},{"osd":10,"last update":"Tue Dec 12 09:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":7,"up_from":50,"seq":214748364833,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1058665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.311,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50900000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.503},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73899999999999999,"5min":0.746,"15min":0.746},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84699999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.72799999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.45600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.871,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83399999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82399999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55800000000000005}]},{"osd":5,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86899999999999999,"5min":1.093,"15min":1.093},"last":0.748},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.83199999999999996,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.497,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":1.113,"15min":1.113},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94499999999999995,"5min":1.018,"15min":1.018},"last":0.64200000000000002}]},{"osd":8,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54100000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81899999999999995}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83499999999999996}]},{"osd":10,"last update":"Tue Dec 12 09:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528357,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":467109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.434}]},{"osd":1,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.60899999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58699999999999997}]},{"osd":3,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.48699999999999999,"15min":0.47499999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.498},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.32200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.44600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.54500000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.68200000000000005,"15min":0.66600000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.51200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.091,"5min":5.069,"15min":5.069},"last":0.72499999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.745,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.56799999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.63300000000000001}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.51600000000000001}]},{"osd":10,"last update":"Tue Dec 12 09:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.751}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":471531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.377,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.51600000000000001,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Dec 12 09:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.59599999999999997,"15min":0.54700000000000004},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":1.032,"15min":1.032},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91400000000000003,"5min":1.079,"15min":1.079},"last":0.85199999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.55100000000000005,"15min":0.52500000000000002},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.57499999999999996,"15min":0.54300000000000004},"min":{"1min":0.46100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.751}]},{"osd":4,"last update":"Tue Dec 12 09:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.41499999999999998,"15min":0.38200000000000001},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.495,"5min":0.42099999999999999,"15min":0.40000000000000002},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58599999999999997,"15min":0.55500000000000005},"min":{"1min":0.40899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":1.018,"15min":1.018},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.59999999999999998,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.61899999999999999,"15min":0.60199999999999998},"min":{"1min":0.44700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89300000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.80400000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":1.042},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":1.012}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.48699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.98799999999999999}]},{"osd":9,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.86799999999999999}]},{"osd":10,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.0269999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":468587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.51900000000000002,"15min":0.48699999999999999},"min":{"1min":0.38900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.52900000000000003,"15min":0.47499999999999998},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.61199999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.56399999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77400000000000002,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.55900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.89300000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.41399999999999998,"15min":0.39500000000000002},"min":{"1min":0.35899999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.436},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.47199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.63400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.016,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.87}]},{"osd":7,"last update":"Tue Dec 12 09:55:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.95999999999999996}]},{"osd":8,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":1.0149999999999999}]},{"osd":9,"last update":"Tue Dec 12 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.98999999999999999}]},{"osd":10,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":1.0329999999999999}]},{"osd":11,"last update":"Tue Dec 12 09:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.0820000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":472590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.34999999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.79200000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.505},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.502,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.754,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72699999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.623}]},{"osd":3,"last update":"Tue Dec 12 09:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.75,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.66900000000000004}]},{"osd":4,"last update":"Tue Dec 12 09:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.54900000000000004,"15min":0.51300000000000001},"min":{"1min":0.47099999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.82599999999999996}]},{"osd":5,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45000000000000001,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.46800000000000003,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52500000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.46700000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.45500000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93600000000000005,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.77700000000000002}]},{"osd":7,"last update":"Tue Dec 12 09:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.81399999999999995}]},{"osd":8,"last update":"Tue Dec 12 09:56:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.877}]},{"osd":10,"last update":"Tue Dec 12 09:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182450,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":467062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.36399999999999999,"15min":0.34699999999999998},"min":{"1min":0.33700000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.37},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.377,"15min":0.35899999999999999},"min":{"1min":0.308,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.40799999999999997,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.55400000000000005,"15min":0.53400000000000003},"min":{"1min":0.48099999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.48199999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.80300000000000005}]},{"osd":4,"last update":"Tue Dec 12 09:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.56200000000000006,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.41099999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60399999999999998,"15min":0.58099999999999996},"min":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.434,"5min":0.432,"15min":0.432},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73499999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.58599999999999997,"15min":0.56399999999999995},"min":{"1min":0.46300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75}]},{"osd":7,"last update":"Tue Dec 12 09:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.51200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.62,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63400000000000001}]},{"osd":8,"last update":"Tue Dec 12 09:55:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58099999999999996}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.84599999999999997}]},{"osd":10,"last update":"Tue Dec 12 09:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953519,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":468197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72899999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.61499999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.25800000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.373}]},{"osd":2,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53700000000000003}]},{"osd":3,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82799999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79400000000000004,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.495}]},{"osd":6,"last update":"Tue Dec 12 09:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.442,"15min":0.442},"max":{"1min":0.871,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59599999999999997}]},{"osd":7,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54600000000000004,"15min":0.53300000000000003},"min":{"1min":0.40699999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89100000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60699999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.53800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92700000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72199999999999998}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70699999999999996}]},{"osd":10,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.621},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1064781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51900000000000002,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.53900000000000003,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.72199999999999998}]},{"osd":2,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.27500000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.39600000000000002},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95199999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.89000000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.439,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58399999999999996,"15min":0.56299999999999994},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.75900000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.53100000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85899999999999999,"5min":1.056,"15min":1.056},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Tue Dec 12 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.496,"5min":0.441,"15min":0.441},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69099999999999995,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.84999999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.432,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.55900000000000005},"min":{"1min":0.375,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58499999999999996}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.79900000000000004}]},{"osd":10,"last update":"Tue Dec 12 09:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":22971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":32760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:57:08.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-12T09:57:08.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 -- 172.21.15.42:0/2916784936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc0ff770 msgr2=0x7fefbc0ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:08.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 --2- 172.21.15.42:0/2916784936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc0ff770 0x7fefbc0ffb90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fefb0009990 tx=0x7fefb0009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 -- 172.21.15.42:0/2916784936 shutdown_connections 2023-12-12T09:57:08.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 --2- 172.21.15.42:0/2916784936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefbc103180 0x7fefbc109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 --2- 172.21.15.42:0/2916784936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc0ff770 0x7fefbc0ffb90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.290 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 --2- 172.21.15.42:0/2916784936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc1027e0 0x7fefbc102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 -- 172.21.15.42:0/2916784936 >> 172.21.15.42:0/2916784936 conn(0x7fefbc0fb0a0 msgr2=0x7fefbc0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 -- 172.21.15.42:0/2916784936 shutdown_connections 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 -- 172.21.15.42:0/2916784936 wait complete. 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 Processor -- start 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.286+0000 7fefc3166700 1 -- start start 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc3166700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc0ff770 0x7fefbc197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc3166700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefbc1027e0 0x7fefbc197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc3166700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc103180 0x7fefbc19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc3166700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc19ef30 con 0x7fefbc103180 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc3166700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc19f070 con 0x7fefbc1027e0 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc3166700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc19f200 con 0x7fefbc0ff770 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc1703700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc103180 0x7fefbc19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc1703700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc103180 0x7fefbc19e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34022/0 (socket says 172.21.15.42:34022) 2023-12-12T09:57:08.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc1703700 1 -- 172.21.15.42:0/2781113288 learned_addr learned my addr 172.21.15.42:0/2781113288 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc0f02700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc0ff770 0x7fefbc197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefbbfff700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefbc1027e0 0x7fefbc197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc1703700 1 -- 172.21.15.42:0/2781113288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc0ff770 msgr2=0x7fefbc197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc1703700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc0ff770 0x7fefbc197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc1703700 1 -- 172.21.15.42:0/2781113288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefbc1027e0 msgr2=0x7fefbc197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc1703700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefbc1027e0 0x7fefbc197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc1703700 1 -- 172.21.15.42:0/2781113288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb00095f0 con 0x7fefbc103180 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc1703700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc103180 0x7fefbc19e960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fefb400ef30 tx=0x7fefb400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefb9ffb700 1 -- 172.21.15.42:0/2781113288 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb4018070 con 0x7fefbc103180 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefb9ffb700 1 -- 172.21.15.42:0/2781113288 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefb400f040 con 0x7fefbc103180 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc19f480 con 0x7fefbc103180 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.287+0000 7fefb9ffb700 1 -- 172.21.15.42:0/2781113288 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb400cd00 con 0x7fefbc103180 2023-12-12T09:57:08.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.288+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc19f990 con 0x7fefbc103180 2023-12-12T09:57:08.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.289+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc108ae0 con 0x7fefbc103180 2023-12-12T09:57:08.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.289+0000 7fefb9ffb700 1 -- 172.21.15.42:0/2781113288 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fefb4003a00 con 0x7fefbc103180 2023-12-12T09:57:08.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.290+0000 7fefb9ffb700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fefac0a1590 0x7fefac0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:08.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.290+0000 7fefb9ffb700 1 -- 172.21.15.42:0/2781113288 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fefb40ccba0 con 0x7fefbc103180 2023-12-12T09:57:08.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.290+0000 7fefc0f02700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fefac0a1590 0x7fefac0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:08.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.291+0000 7fefc0f02700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fefac0a1590 0x7fefac0a3a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefb000b680 tx=0x7fefb0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:08.297 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.295+0000 7fefb9ffb700 1 -- 172.21.15.42:0/2781113288 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fefb4095f40 con 0x7fefbc103180 2023-12-12T09:57:08.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:08 smithi042 ceph-mon[98894]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:08.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:08 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] up:active 2023-12-12T09:57:08.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:08 smithi042 ceph-mon[98894]: fsmap cephfs:5 7 up:standby 5 up:active 2023-12-12T09:57:08.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:08 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T09:57:08.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:08 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2381901367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:08.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:08 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1696753927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:08.495 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.493+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fefbc061a10 con 0x7fefac0a1590 2023-12-12T09:57:08.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:08 smithi111 ceph-mon[102565]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:08.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:08 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] up:active 2023-12-12T09:57:08.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:08 smithi111 ceph-mon[102565]: fsmap cephfs:5 7 up:standby 5 up:active 2023-12-12T09:57:08.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:08 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T09:57:08.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:08 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2381901367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:08.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:08 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1696753927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:08.498 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:57:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: 2023-12-12T09:57:08.244+0000 7fbe9c1df700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2023-12-12T09:57:08.498 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:57:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: 2023-12-12T09:57:08.244+0000 7fbe9c1df700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2023-12-12T09:57:08.498 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 09:57:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: 2023-12-12T09:57:08.244+0000 7fbe9c1df700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2023-12-12T09:57:08.510 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.508+0000 7fefb9ffb700 1 -- 172.21.15.42:0/2781113288 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423083 (secure 0 0 0) 0x7fefbc061a10 con 0x7fefac0a1590 2023-12-12T09:57:08.515 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fefac0a1590 msgr2=0x7fefac0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fefac0a1590 0x7fefac0a3a50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefb000b680 tx=0x7fefb0005fd0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.518 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc103180 msgr2=0x7fefbc19e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc103180 0x7fefbc19e960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fefb400ef30 tx=0x7fefb400c3a0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 shutdown_connections 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fefac0a1590 0x7fefac0a3a50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fefbc103180 0x7fefbc19e960 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fefbc0ff770 0x7fefbc197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 --2- 172.21.15.42:0/2781113288 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fefbc1027e0 0x7fefbc197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 >> 172.21.15.42:0/2781113288 conn(0x7fefbc0fb0a0 msgr2=0x7fefbc105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 shutdown_connections 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:08.512+0000 7fefc3166700 1 -- 172.21.15.42:0/2781113288 wait complete. 2023-12-12T09:57:08.519 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2023-12-12T09:57:08.527 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2023-12-12T09:57:07.814046+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600649,"num_objects":80,"num_object_clones":0,"num_object_copies":241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":0,"num_read":152,"num_read_kb":134,"num_write":345,"num_write_kb":2997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":284,"ondisk_log_size":284,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":337432,"kb_used_data":13236,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124604392,"statfs":{"total":1151940427776,"available":1151594897408,"internally_reserved":0,"allocated":13553664,"data_stored":7408787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5784,"num_objects":56,"num_object_clones":0,"num_object_copies":169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":117,"num_write_kb":42,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003719"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494114+0000","last_change":"2023-12-12T09:56:38.273952+0000","last_active":"2023-12-12T09:56:42.494114+0000","last_peered":"2023-12-12T09:56:42.494114+0000","last_clean":"2023-12-12T09:56:42.494114+0000","last_became_active":"2023-12-12T09:56:38.273609+0000","last_became_peered":"2023-12-12T09:56:38.273609+0000","last_unstale":"2023-12-12T09:56:42.494114+0000","last_undegraded":"2023-12-12T09:56:42.494114+0000","last_fullsized":"2023-12-12T09:56:42.494114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:23:43.144987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496587+0000","last_change":"2023-12-12T09:56:38.278719+0000","last_active":"2023-12-12T09:56:42.496587+0000","last_peered":"2023-12-12T09:56:42.496587+0000","last_clean":"2023-12-12T09:56:42.496587+0000","last_became_active":"2023-12-12T09:56:38.278404+0000","last_became_peered":"2023-12-12T09:56:38.278404+0000","last_unstale":"2023-12-12T09:56:42.496587+0000","last_undegraded":"2023-12-12T09:56:42.496587+0000","last_fullsized":"2023-12-12T09:56:42.496587+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:34:01.139106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494931+0000","last_change":"2023-12-12T09:56:38.274276+0000","last_active":"2023-12-12T09:56:42.494931+0000","last_peered":"2023-12-12T09:56:42.494931+0000","last_clean":"2023-12-12T09:56:42.494931+0000","last_became_active":"2023-12-12T09:56:38.273974+0000","last_became_peered":"2023-12-12T09:56:38.273974+0000","last_unstale":"2023-12-12T09:56:42.494931+0000","last_undegraded":"2023-12-12T09:56:42.494931+0000","last_fullsized":"2023-12-12T09:56:42.494931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:40:55.979238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.490962+0000","last_change":"2023-12-12T09:56:38.276021+0000","last_active":"2023-12-12T09:56:42.490962+0000","last_peered":"2023-12-12T09:56:42.490962+0000","last_clean":"2023-12-12T09:56:42.490962+0000","last_became_active":"2023-12-12T09:56:38.275305+0000","last_became_peered":"2023-12-12T09:56:38.275305+0000","last_unstale":"2023-12-12T09:56:42.490962+0000","last_undegraded":"2023-12-12T09:56:42.490962+0000","last_fullsized":"2023-12-12T09:56:42.490962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:32:28.839430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.491750+0000","last_change":"2023-12-12T09:56:38.286720+0000","last_active":"2023-12-12T09:56:42.491750+0000","last_peered":"2023-12-12T09:56:42.491750+0000","last_clean":"2023-12-12T09:56:42.491750+0000","last_became_active":"2023-12-12T09:56:38.286410+0000","last_became_peered":"2023-12-12T09:56:38.286410+0000","last_unstale":"2023-12-12T09:56:42.491750+0000","last_undegraded":"2023-12-12T09:56:42.491750+0000","last_fullsized":"2023-12-12T09:56:42.491750+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:07:42.131132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576251+0000","last_change":"2023-12-12T09:56:38.273350+0000","last_active":"2023-12-12T09:56:42.576251+0000","last_peered":"2023-12-12T09:56:42.576251+0000","last_clean":"2023-12-12T09:56:42.576251+0000","last_became_active":"2023-12-12T09:56:38.273077+0000","last_became_peered":"2023-12-12T09:56:38.273077+0000","last_unstale":"2023-12-12T09:56:42.576251+0000","last_undegraded":"2023-12-12T09:56:42.576251+0000","last_fullsized":"2023-12-12T09:56:42.576251+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:28.081132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496617+0000","last_change":"2023-12-12T09:56:38.278212+0000","last_active":"2023-12-12T09:56:42.496617+0000","last_peered":"2023-12-12T09:56:42.496617+0000","last_clean":"2023-12-12T09:56:42.496617+0000","last_became_active":"2023-12-12T09:56:38.277913+0000","last_became_peered":"2023-12-12T09:56:38.277913+0000","last_unstale":"2023-12-12T09:56:42.496617+0000","last_undegraded":"2023-12-12T09:56:42.496617+0000","last_fullsized":"2023-12-12T09:56:42.496617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:39:11.893381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.486676+0000","last_change":"2023-12-12T09:56:38.286625+0000","last_active":"2023-12-12T09:56:42.486676+0000","last_peered":"2023-12-12T09:56:42.486676+0000","last_clean":"2023-12-12T09:56:42.486676+0000","last_became_active":"2023-12-12T09:56:38.286035+0000","last_became_peered":"2023-12-12T09:56:38.286035+0000","last_unstale":"2023-12-12T09:56:42.486676+0000","last_undegraded":"2023-12-12T09:56:42.486676+0000","last_fullsized":"2023-12-12T09:56:42.486676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:13:03.723529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495016+0000","last_change":"2023-12-12T09:56:38.281533+0000","last_active":"2023-12-12T09:56:42.495016+0000","last_peered":"2023-12-12T09:56:42.495016+0000","last_clean":"2023-12-12T09:56:42.495016+0000","last_became_active":"2023-12-12T09:56:38.281228+0000","last_became_peered":"2023-12-12T09:56:38.281228+0000","last_unstale":"2023-12-12T09:56:42.495016+0000","last_undegraded":"2023-12-12T09:56:42.495016+0000","last_fullsized":"2023-12-12T09:56:42.495016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:49:34.075982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576986+0000","last_change":"2023-12-12T09:56:38.274599+0000","last_active":"2023-12-12T09:56:42.576986+0000","last_peered":"2023-12-12T09:56:42.576986+0000","last_clean":"2023-12-12T09:56:42.576986+0000","last_became_active":"2023-12-12T09:56:38.274309+0000","last_became_peered":"2023-12-12T09:56:38.274309+0000","last_unstale":"2023-12-12T09:56:42.576986+0000","last_undegraded":"2023-12-12T09:56:42.576986+0000","last_fullsized":"2023-12-12T09:56:42.576986+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:05:06.054345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497435+0000","last_change":"2023-12-12T09:56:38.279348+0000","last_active":"2023-12-12T09:56:42.497435+0000","last_peered":"2023-12-12T09:56:42.497435+0000","last_clean":"2023-12-12T09:56:42.497435+0000","last_became_active":"2023-12-12T09:56:38.278843+0000","last_became_peered":"2023-12-12T09:56:38.278843+0000","last_unstale":"2023-12-12T09:56:42.497435+0000","last_undegraded":"2023-12-12T09:56:42.497435+0000","last_fullsized":"2023-12-12T09:56:42.497435+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:50:04.248976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.490866+0000","last_change":"2023-12-12T09:56:38.276052+0000","last_active":"2023-12-12T09:56:42.490866+0000","last_peered":"2023-12-12T09:56:42.490866+0000","last_clean":"2023-12-12T09:56:42.490866+0000","last_became_active":"2023-12-12T09:56:38.275712+0000","last_became_peered":"2023-12-12T09:56:38.275712+0000","last_unstale":"2023-12-12T09:56:42.490866+0000","last_undegraded":"2023-12-12T09:56:42.490866+0000","last_fullsized":"2023-12-12T09:56:42.490866+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:37:06.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499766+0000","last_change":"2023-12-12T09:56:38.284052+0000","last_active":"2023-12-12T09:56:42.499766+0000","last_peered":"2023-12-12T09:56:42.499766+0000","last_clean":"2023-12-12T09:56:42.499766+0000","last_became_active":"2023-12-12T09:56:38.283741+0000","last_became_peered":"2023-12-12T09:56:38.283741+0000","last_unstale":"2023-12-12T09:56:42.499766+0000","last_undegraded":"2023-12-12T09:56:42.499766+0000","last_fullsized":"2023-12-12T09:56:42.499766+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:40:47.505972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494574+0000","last_change":"2023-12-12T09:56:38.281604+0000","last_active":"2023-12-12T09:56:42.494574+0000","last_peered":"2023-12-12T09:56:42.494574+0000","last_clean":"2023-12-12T09:56:42.494574+0000","last_became_active":"2023-12-12T09:56:38.281247+0000","last_became_peered":"2023-12-12T09:56:38.281247+0000","last_unstale":"2023-12-12T09:56:42.494574+0000","last_undegraded":"2023-12-12T09:56:42.494574+0000","last_fullsized":"2023-12-12T09:56:42.494574+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:36:14.002676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495303+0000","last_change":"2023-12-12T09:56:38.273599+0000","last_active":"2023-12-12T09:56:42.495303+0000","last_peered":"2023-12-12T09:56:42.495303+0000","last_clean":"2023-12-12T09:56:42.495303+0000","last_became_active":"2023-12-12T09:56:38.273290+0000","last_became_peered":"2023-12-12T09:56:38.273290+0000","last_unstale":"2023-12-12T09:56:42.495303+0000","last_undegraded":"2023-12-12T09:56:42.495303+0000","last_fullsized":"2023-12-12T09:56:42.495303+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:54:52.088314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495603+0000","last_change":"2023-12-12T09:56:38.275663+0000","last_active":"2023-12-12T09:56:42.495603+0000","last_peered":"2023-12-12T09:56:42.495603+0000","last_clean":"2023-12-12T09:56:42.495603+0000","last_became_active":"2023-12-12T09:56:38.275322+0000","last_became_peered":"2023-12-12T09:56:38.275322+0000","last_unstale":"2023-12-12T09:56:42.495603+0000","last_undegraded":"2023-12-12T09:56:42.495603+0000","last_fullsized":"2023-12-12T09:56:42.495603+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:46:50.067328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494023+0000","last_change":"2023-12-12T09:56:38.274064+0000","last_active":"2023-12-12T09:56:42.494023+0000","last_peered":"2023-12-12T09:56:42.494023+0000","last_clean":"2023-12-12T09:56:42.494023+0000","last_became_active":"2023-12-12T09:56:38.273733+0000","last_became_peered":"2023-12-12T09:56:38.273733+0000","last_unstale":"2023-12-12T09:56:42.494023+0000","last_undegraded":"2023-12-12T09:56:42.494023+0000","last_fullsized":"2023-12-12T09:56:42.494023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:20:31.662432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.501075+0000","last_change":"2023-12-12T09:56:38.277627+0000","last_active":"2023-12-12T09:56:42.501075+0000","last_peered":"2023-12-12T09:56:42.501075+0000","last_clean":"2023-12-12T09:56:42.501075+0000","last_became_active":"2023-12-12T09:56:38.277275+0000","last_became_peered":"2023-12-12T09:56:38.277275+0000","last_unstale":"2023-12-12T09:56:42.501075+0000","last_undegraded":"2023-12-12T09:56:42.501075+0000","last_fullsized":"2023-12-12T09:56:42.501075+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:37:57.256691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496895+0000","last_change":"2023-12-12T09:56:38.276900+0000","last_active":"2023-12-12T09:56:42.496895+0000","last_peered":"2023-12-12T09:56:42.496895+0000","last_clean":"2023-12-12T09:56:42.496895+0000","last_became_active":"2023-12-12T09:56:38.276384+0000","last_became_peered":"2023-12-12T09:56:38.276384+0000","last_unstale":"2023-12-12T09:56:42.496895+0000","last_undegraded":"2023-12-12T09:56:42.496895+0000","last_fullsized":"2023-12-12T09:56:42.496895+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:13:37.097349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.554898+0000","last_change":"2023-12-12T09:56:42.509239+0000","last_active":"2023-12-12T09:56:43.554898+0000","last_peered":"2023-12-12T09:56:43.554898+0000","last_clean":"2023-12-12T09:56:43.554898+0000","last_became_active":"2023-12-12T09:56:42.508699+0000","last_became_peered":"2023-12-12T09:56:42.508699+0000","last_unstale":"2023-12-12T09:56:43.554898+0000","last_undegraded":"2023-12-12T09:56:43.554898+0000","last_fullsized":"2023-12-12T09:56:43.554898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:12:39.501023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:04.288334+0000","last_change":"2023-12-12T09:56:37.476258+0000","last_active":"2023-12-12T09:57:04.288334+0000","last_peered":"2023-12-12T09:57:04.288334+0000","last_clean":"2023-12-12T09:57:04.288334+0000","last_became_active":"2023-12-12T09:56:37.475886+0000","last_became_peered":"2023-12-12T09:56:37.475886+0000","last_unstale":"2023-12-12T09:57:04.288334+0000","last_undegraded":"2023-12-12T09:57:04.288334+0000","last_fullsized":"2023-12-12T09:57:04.288334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:29:34.770549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.486999+0000","last_change":"2023-12-12T09:56:38.284968+0000","last_active":"2023-12-12T09:56:42.486999+0000","last_peered":"2023-12-12T09:56:42.486999+0000","last_clean":"2023-12-12T09:56:42.486999+0000","last_became_active":"2023-12-12T09:56:38.284661+0000","last_became_peered":"2023-12-12T09:56:38.284661+0000","last_unstale":"2023-12-12T09:56:42.486999+0000","last_undegraded":"2023-12-12T09:56:42.486999+0000","last_fullsized":"2023-12-12T09:56:42.486999+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:27:02.489220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681309+0000","last_change":"2023-12-12T09:56:42.512514+0000","last_active":"2023-12-12T09:56:43.681309+0000","last_peered":"2023-12-12T09:56:43.681309+0000","last_clean":"2023-12-12T09:56:43.681309+0000","last_became_active":"2023-12-12T09:56:42.512217+0000","last_became_peered":"2023-12-12T09:56:42.512217+0000","last_unstale":"2023-12-12T09:56:43.681309+0000","last_undegraded":"2023-12-12T09:56:43.681309+0000","last_fullsized":"2023-12-12T09:56:43.681309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:46:28.422801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680885+0000","last_change":"2023-12-12T09:56:37.471047+0000","last_active":"2023-12-12T09:56:43.680885+0000","last_peered":"2023-12-12T09:56:43.680885+0000","last_clean":"2023-12-12T09:56:43.680885+0000","last_became_active":"2023-12-12T09:56:37.470689+0000","last_became_peered":"2023-12-12T09:56:37.470689+0000","last_unstale":"2023-12-12T09:56:43.680885+0000","last_undegraded":"2023-12-12T09:56:43.680885+0000","last_fullsized":"2023-12-12T09:56:43.680885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:00.648165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496361+0000","last_change":"2023-12-12T09:56:38.275884+0000","last_active":"2023-12-12T09:56:42.496361+0000","last_peered":"2023-12-12T09:56:42.496361+0000","last_clean":"2023-12-12T09:56:42.496361+0000","last_became_active":"2023-12-12T09:56:38.275161+0000","last_became_peered":"2023-12-12T09:56:38.275161+0000","last_unstale":"2023-12-12T09:56:42.496361+0000","last_undegraded":"2023-12-12T09:56:42.496361+0000","last_fullsized":"2023-12-12T09:56:42.496361+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:06:02.863719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682109+0000","last_change":"2023-12-12T09:56:42.507539+0000","last_active":"2023-12-12T09:56:43.682109+0000","last_peered":"2023-12-12T09:56:43.682109+0000","last_clean":"2023-12-12T09:56:43.682109+0000","last_became_active":"2023-12-12T09:56:42.507008+0000","last_became_peered":"2023-12-12T09:56:42.507008+0000","last_unstale":"2023-12-12T09:56:43.682109+0000","last_undegraded":"2023-12-12T09:56:43.682109+0000","last_fullsized":"2023-12-12T09:56:43.682109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:28:30.449096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'11","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.302249+0000","last_change":"2023-12-12T09:56:37.475002+0000","last_active":"2023-12-12T09:57:06.302249+0000","last_peered":"2023-12-12T09:57:06.302249+0000","last_clean":"2023-12-12T09:57:06.302249+0000","last_became_active":"2023-12-12T09:56:37.474675+0000","last_became_peered":"2023-12-12T09:56:37.474675+0000","last_unstale":"2023-12-12T09:57:06.302249+0000","last_undegraded":"2023-12-12T09:57:06.302249+0000","last_fullsized":"2023-12-12T09:57:06.302249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:38:15.880299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.493588+0000","last_change":"2023-12-12T09:56:38.280532+0000","last_active":"2023-12-12T09:56:42.493588+0000","last_peered":"2023-12-12T09:56:42.493588+0000","last_clean":"2023-12-12T09:56:42.493588+0000","last_became_active":"2023-12-12T09:56:38.280177+0000","last_became_peered":"2023-12-12T09:56:38.280177+0000","last_unstale":"2023-12-12T09:56:42.493588+0000","last_undegraded":"2023-12-12T09:56:42.493588+0000","last_fullsized":"2023-12-12T09:56:42.493588+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:02:34.818978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681062+0000","last_change":"2023-12-12T09:56:42.504506+0000","last_active":"2023-12-12T09:56:43.681062+0000","last_peered":"2023-12-12T09:56:43.681062+0000","last_clean":"2023-12-12T09:56:43.681062+0000","last_became_active":"2023-12-12T09:56:42.504104+0000","last_became_peered":"2023-12-12T09:56:42.504104+0000","last_unstale":"2023-12-12T09:56:43.681062+0000","last_undegraded":"2023-12-12T09:56:43.681062+0000","last_fullsized":"2023-12-12T09:56:43.681062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:12:41.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.301432+0000","last_change":"2023-12-12T09:56:37.475222+0000","last_active":"2023-12-12T09:57:06.301432+0000","last_peered":"2023-12-12T09:57:06.301432+0000","last_clean":"2023-12-12T09:57:06.301432+0000","last_became_active":"2023-12-12T09:56:37.474917+0000","last_became_peered":"2023-12-12T09:56:37.474917+0000","last_unstale":"2023-12-12T09:57:06.301432+0000","last_undegraded":"2023-12-12T09:57:06.301432+0000","last_fullsized":"2023-12-12T09:57:06.301432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:26:36.639215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487408+0000","last_change":"2023-12-12T09:56:38.277949+0000","last_active":"2023-12-12T09:56:42.487408+0000","last_peered":"2023-12-12T09:56:42.487408+0000","last_clean":"2023-12-12T09:56:42.487408+0000","last_became_active":"2023-12-12T09:56:38.277625+0000","last_became_peered":"2023-12-12T09:56:38.277625+0000","last_unstale":"2023-12-12T09:56:42.487408+0000","last_undegraded":"2023-12-12T09:56:42.487408+0000","last_fullsized":"2023-12-12T09:56:42.487408+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:42:46.906101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.555152+0000","last_change":"2023-12-12T09:56:42.502196+0000","last_active":"2023-12-12T09:56:43.555152+0000","last_peered":"2023-12-12T09:56:43.555152+0000","last_clean":"2023-12-12T09:56:43.555152+0000","last_became_active":"2023-12-12T09:56:42.501789+0000","last_became_peered":"2023-12-12T09:56:42.501789+0000","last_unstale":"2023-12-12T09:56:43.555152+0000","last_undegraded":"2023-12-12T09:56:43.555152+0000","last_fullsized":"2023-12-12T09:56:43.555152+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:04:02.656074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:07.302672+0000","last_change":"2023-12-12T09:56:37.472404+0000","last_active":"2023-12-12T09:57:07.302672+0000","last_peered":"2023-12-12T09:57:07.302672+0000","last_clean":"2023-12-12T09:57:07.302672+0000","last_became_active":"2023-12-12T09:56:37.472063+0000","last_became_peered":"2023-12-12T09:56:37.472063+0000","last_unstale":"2023-12-12T09:57:07.302672+0000","last_undegraded":"2023-12-12T09:57:07.302672+0000","last_fullsized":"2023-12-12T09:57:07.302672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:14:07.020310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.492300+0000","last_change":"2023-12-12T09:56:38.281152+0000","last_active":"2023-12-12T09:56:42.492300+0000","last_peered":"2023-12-12T09:56:42.492300+0000","last_clean":"2023-12-12T09:56:42.492300+0000","last_became_active":"2023-12-12T09:56:38.280793+0000","last_became_peered":"2023-12-12T09:56:38.280793+0000","last_unstale":"2023-12-12T09:56:42.492300+0000","last_undegraded":"2023-12-12T09:56:42.492300+0000","last_fullsized":"2023-12-12T09:56:42.492300+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.554920+0000","last_change":"2023-12-12T09:56:42.507439+0000","last_active":"2023-12-12T09:56:43.554920+0000","last_peered":"2023-12-12T09:56:43.554920+0000","last_clean":"2023-12-12T09:56:43.554920+0000","last_became_active":"2023-12-12T09:56:42.506980+0000","last_became_peered":"2023-12-12T09:56:42.506980+0000","last_unstale":"2023-12-12T09:56:43.554920+0000","last_undegraded":"2023-12-12T09:56:43.554920+0000","last_fullsized":"2023-12-12T09:56:43.554920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:37.086804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"82'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576251+0000","last_change":"2023-12-12T09:56:37.473666+0000","last_active":"2023-12-12T09:56:42.576251+0000","last_peered":"2023-12-12T09:56:42.576251+0000","last_clean":"2023-12-12T09:56:42.576251+0000","last_became_active":"2023-12-12T09:56:37.473349+0000","last_became_peered":"2023-12-12T09:56:37.473349+0000","last_unstale":"2023-12-12T09:56:42.576251+0000","last_undegraded":"2023-12-12T09:56:42.576251+0000","last_fullsized":"2023-12-12T09:56:42.576251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.590517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487307+0000","last_change":"2023-12-12T09:56:38.279220+0000","last_active":"2023-12-12T09:56:42.487307+0000","last_peered":"2023-12-12T09:56:42.487307+0000","last_clean":"2023-12-12T09:56:42.487307+0000","last_became_active":"2023-12-12T09:56:38.278950+0000","last_became_peered":"2023-12-12T09:56:38.278950+0000","last_unstale":"2023-12-12T09:56:42.487307+0000","last_undegraded":"2023-12-12T09:56:42.487307+0000","last_fullsized":"2023-12-12T09:56:42.487307+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:06:24.247294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680871+0000","last_change":"2023-12-12T09:56:42.507022+0000","last_active":"2023-12-12T09:56:43.680871+0000","last_peered":"2023-12-12T09:56:43.680871+0000","last_clean":"2023-12-12T09:56:43.680871+0000","last_became_active":"2023-12-12T09:56:42.506575+0000","last_became_peered":"2023-12-12T09:56:42.506575+0000","last_unstale":"2023-12-12T09:56:43.680871+0000","last_undegraded":"2023-12-12T09:56:43.680871+0000","last_fullsized":"2023-12-12T09:56:43.680871+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:46.685081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499372+0000","last_change":"2023-12-12T09:56:37.464083+0000","last_active":"2023-12-12T09:56:42.499372+0000","last_peered":"2023-12-12T09:56:42.499372+0000","last_clean":"2023-12-12T09:56:42.499372+0000","last_became_active":"2023-12-12T09:56:37.463774+0000","last_became_peered":"2023-12-12T09:56:37.463774+0000","last_unstale":"2023-12-12T09:56:42.499372+0000","last_undegraded":"2023-12-12T09:56:42.499372+0000","last_fullsized":"2023-12-12T09:56:42.499372+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:34.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.486804+0000","last_change":"2023-12-12T09:56:38.278729+0000","last_active":"2023-12-12T09:56:42.486804+0000","last_peered":"2023-12-12T09:56:42.486804+0000","last_clean":"2023-12-12T09:56:42.486804+0000","last_became_active":"2023-12-12T09:56:38.278193+0000","last_became_peered":"2023-12-12T09:56:38.278193+0000","last_unstale":"2023-12-12T09:56:42.486804+0000","last_undegraded":"2023-12-12T09:56:42.486804+0000","last_fullsized":"2023-12-12T09:56:42.486804+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:20:25.854114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.262476+0000","last_change":"2023-12-12T09:56:42.504228+0000","last_active":"2023-12-12T09:56:43.262476+0000","last_peered":"2023-12-12T09:56:43.262476+0000","last_clean":"2023-12-12T09:56:43.262476+0000","last_became_active":"2023-12-12T09:56:42.503823+0000","last_became_peered":"2023-12-12T09:56:42.503823+0000","last_unstale":"2023-12-12T09:56:43.262476+0000","last_undegraded":"2023-12-12T09:56:43.262476+0000","last_fullsized":"2023-12-12T09:56:43.262476+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:54:05.263784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.300013+0000","last_change":"2023-12-12T09:56:37.472108+0000","last_active":"2023-12-12T09:57:06.300013+0000","last_peered":"2023-12-12T09:57:06.300013+0000","last_clean":"2023-12-12T09:57:06.300013+0000","last_became_active":"2023-12-12T09:56:37.471822+0000","last_became_peered":"2023-12-12T09:56:37.471822+0000","last_unstale":"2023-12-12T09:57:06.300013+0000","last_undegraded":"2023-12-12T09:57:06.300013+0000","last_fullsized":"2023-12-12T09:57:06.300013+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:19:24.432739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495125+0000","last_change":"2023-12-12T09:56:38.274724+0000","last_active":"2023-12-12T09:56:42.495125+0000","last_peered":"2023-12-12T09:56:42.495125+0000","last_clean":"2023-12-12T09:56:42.495125+0000","last_became_active":"2023-12-12T09:56:38.274383+0000","last_became_peered":"2023-12-12T09:56:38.274383+0000","last_unstale":"2023-12-12T09:56:42.495125+0000","last_undegraded":"2023-12-12T09:56:42.495125+0000","last_fullsized":"2023-12-12T09:56:42.495125+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:21:37.512880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680866+0000","last_change":"2023-12-12T09:56:42.505510+0000","last_active":"2023-12-12T09:56:43.680866+0000","last_peered":"2023-12-12T09:56:43.680866+0000","last_clean":"2023-12-12T09:56:43.680866+0000","last_became_active":"2023-12-12T09:56:42.504141+0000","last_became_peered":"2023-12-12T09:56:42.504141+0000","last_unstale":"2023-12-12T09:56:43.680866+0000","last_undegraded":"2023-12-12T09:56:43.680866+0000","last_fullsized":"2023-12-12T09:56:43.680866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:48:00.108207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.279949+0000","last_change":"2023-12-12T09:56:37.466706+0000","last_active":"2023-12-12T09:57:03.279949+0000","last_peered":"2023-12-12T09:57:03.279949+0000","last_clean":"2023-12-12T09:57:03.279949+0000","last_became_active":"2023-12-12T09:56:37.466268+0000","last_became_peered":"2023-12-12T09:56:37.466268+0000","last_unstale":"2023-12-12T09:57:03.279949+0000","last_undegraded":"2023-12-12T09:57:03.279949+0000","last_fullsized":"2023-12-12T09:57:03.279949+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:54:50.868616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.493874+0000","last_change":"2023-12-12T09:56:38.277134+0000","last_active":"2023-12-12T09:56:42.493874+0000","last_peered":"2023-12-12T09:56:42.493874+0000","last_clean":"2023-12-12T09:56:42.493874+0000","last_became_active":"2023-12-12T09:56:38.276701+0000","last_became_peered":"2023-12-12T09:56:38.276701+0000","last_unstale":"2023-12-12T09:56:42.493874+0000","last_undegraded":"2023-12-12T09:56:42.493874+0000","last_fullsized":"2023-12-12T09:56:42.493874+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:46:02.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681833+0000","last_change":"2023-12-12T09:56:42.506494+0000","last_active":"2023-12-12T09:56:43.681833+0000","last_peered":"2023-12-12T09:56:43.681833+0000","last_clean":"2023-12-12T09:56:43.681833+0000","last_became_active":"2023-12-12T09:56:42.505965+0000","last_became_peered":"2023-12-12T09:56:42.505965+0000","last_unstale":"2023-12-12T09:56:43.681833+0000","last_undegraded":"2023-12-12T09:56:43.681833+0000","last_fullsized":"2023-12-12T09:56:43.681833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:32:01.294474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.275430+0000","last_change":"2023-12-12T09:56:37.469046+0000","last_active":"2023-12-12T09:57:03.275430+0000","last_peered":"2023-12-12T09:57:03.275430+0000","last_clean":"2023-12-12T09:57:03.275430+0000","last_became_active":"2023-12-12T09:56:37.468112+0000","last_became_peered":"2023-12-12T09:56:37.468112+0000","last_unstale":"2023-12-12T09:57:03.275430+0000","last_undegraded":"2023-12-12T09:57:03.275430+0000","last_fullsized":"2023-12-12T09:57:03.275430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:39:35.356243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496268+0000","last_change":"2023-12-12T09:56:38.280000+0000","last_active":"2023-12-12T09:56:42.496268+0000","last_peered":"2023-12-12T09:56:42.496268+0000","last_clean":"2023-12-12T09:56:42.496268+0000","last_became_active":"2023-12-12T09:56:38.279723+0000","last_became_peered":"2023-12-12T09:56:38.279723+0000","last_unstale":"2023-12-12T09:56:42.496268+0000","last_undegraded":"2023-12-12T09:56:42.496268+0000","last_fullsized":"2023-12-12T09:56:42.496268+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:56:35.326614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682388+0000","last_change":"2023-12-12T09:56:42.506820+0000","last_active":"2023-12-12T09:56:43.682388+0000","last_peered":"2023-12-12T09:56:43.682388+0000","last_clean":"2023-12-12T09:56:43.682388+0000","last_became_active":"2023-12-12T09:56:42.506481+0000","last_became_peered":"2023-12-12T09:56:42.506481+0000","last_unstale":"2023-12-12T09:56:43.682388+0000","last_undegraded":"2023-12-12T09:56:43.682388+0000","last_fullsized":"2023-12-12T09:56:43.682388+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:10:50.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.299662+0000","last_change":"2023-12-12T09:56:37.470283+0000","last_active":"2023-12-12T09:57:05.299662+0000","last_peered":"2023-12-12T09:57:05.299662+0000","last_clean":"2023-12-12T09:57:05.299662+0000","last_became_active":"2023-12-12T09:56:37.469568+0000","last_became_peered":"2023-12-12T09:56:37.469568+0000","last_unstale":"2023-12-12T09:57:05.299662+0000","last_undegraded":"2023-12-12T09:57:05.299662+0000","last_fullsized":"2023-12-12T09:57:05.299662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:24.427658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.493857+0000","last_change":"2023-12-12T09:56:38.279180+0000","last_active":"2023-12-12T09:56:42.493857+0000","last_peered":"2023-12-12T09:56:42.493857+0000","last_clean":"2023-12-12T09:56:42.493857+0000","last_became_active":"2023-12-12T09:56:38.278493+0000","last_became_peered":"2023-12-12T09:56:38.278493+0000","last_unstale":"2023-12-12T09:56:42.493857+0000","last_undegraded":"2023-12-12T09:56:42.493857+0000","last_fullsized":"2023-12-12T09:56:42.493857+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:17:52.468663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682004+0000","last_change":"2023-12-12T09:56:42.505140+0000","last_active":"2023-12-12T09:56:43.682004+0000","last_peered":"2023-12-12T09:56:43.682004+0000","last_clean":"2023-12-12T09:56:43.682004+0000","last_became_active":"2023-12-12T09:56:42.504655+0000","last_became_peered":"2023-12-12T09:56:42.504655+0000","last_unstale":"2023-12-12T09:56:43.682004+0000","last_undegraded":"2023-12-12T09:56:43.682004+0000","last_fullsized":"2023-12-12T09:56:43.682004+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:58:18.004084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"87'12","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.279365+0000","last_change":"2023-12-12T09:56:37.468048+0000","last_active":"2023-12-12T09:57:03.279365+0000","last_peered":"2023-12-12T09:57:03.279365+0000","last_clean":"2023-12-12T09:57:03.279365+0000","last_became_active":"2023-12-12T09:56:37.467744+0000","last_became_peered":"2023-12-12T09:56:37.467744+0000","last_unstale":"2023-12-12T09:57:03.279365+0000","last_undegraded":"2023-12-12T09:57:03.279365+0000","last_fullsized":"2023-12-12T09:57:03.279365+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:29:56.033230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3840,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.492676+0000","last_change":"2023-12-12T09:56:38.284640+0000","last_active":"2023-12-12T09:56:42.492676+0000","last_peered":"2023-12-12T09:56:42.492676+0000","last_clean":"2023-12-12T09:56:42.492676+0000","last_became_active":"2023-12-12T09:56:38.284351+0000","last_became_peered":"2023-12-12T09:56:38.284351+0000","last_unstale":"2023-12-12T09:56:42.492676+0000","last_undegraded":"2023-12-12T09:56:42.492676+0000","last_fullsized":"2023-12-12T09:56:42.492676+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:22.910616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682133+0000","last_change":"2023-12-12T09:56:42.502821+0000","last_active":"2023-12-12T09:56:43.682133+0000","last_peered":"2023-12-12T09:56:43.682133+0000","last_clean":"2023-12-12T09:56:43.682133+0000","last_became_active":"2023-12-12T09:56:42.502377+0000","last_became_peered":"2023-12-12T09:56:42.502377+0000","last_unstale":"2023-12-12T09:56:43.682133+0000","last_undegraded":"2023-12-12T09:56:43.682133+0000","last_fullsized":"2023-12-12T09:56:43.682133+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:41:22.913984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.301174+0000","last_change":"2023-12-12T09:56:37.468656+0000","last_active":"2023-12-12T09:57:06.301174+0000","last_peered":"2023-12-12T09:57:06.301174+0000","last_clean":"2023-12-12T09:57:06.301174+0000","last_became_active":"2023-12-12T09:56:37.468353+0000","last_became_peered":"2023-12-12T09:56:37.468353+0000","last_unstale":"2023-12-12T09:57:06.301174+0000","last_undegraded":"2023-12-12T09:57:06.301174+0000","last_fullsized":"2023-12-12T09:57:06.301174+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:51:21.477423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.500156+0000","last_change":"2023-12-12T09:56:38.277290+0000","last_active":"2023-12-12T09:56:42.500156+0000","last_peered":"2023-12-12T09:56:42.500156+0000","last_clean":"2023-12-12T09:56:42.500156+0000","last_became_active":"2023-12-12T09:56:38.276972+0000","last_became_peered":"2023-12-12T09:56:38.276972+0000","last_unstale":"2023-12-12T09:56:42.500156+0000","last_undegraded":"2023-12-12T09:56:42.500156+0000","last_fullsized":"2023-12-12T09:56:42.500156+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:03:25.254368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680672+0000","last_change":"2023-12-12T09:56:42.506900+0000","last_active":"2023-12-12T09:56:43.680672+0000","last_peered":"2023-12-12T09:56:43.680672+0000","last_clean":"2023-12-12T09:56:43.680672+0000","last_became_active":"2023-12-12T09:56:42.506470+0000","last_became_peered":"2023-12-12T09:56:42.506470+0000","last_unstale":"2023-12-12T09:56:43.680672+0000","last_undegraded":"2023-12-12T09:56:43.680672+0000","last_fullsized":"2023-12-12T09:56:43.680672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:04:17.359766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.299212+0000","last_change":"2023-12-12T09:56:37.472598+0000","last_active":"2023-12-12T09:57:06.299212+0000","last_peered":"2023-12-12T09:57:06.299212+0000","last_clean":"2023-12-12T09:57:06.299212+0000","last_became_active":"2023-12-12T09:56:37.471809+0000","last_became_peered":"2023-12-12T09:56:37.471809+0000","last_unstale":"2023-12-12T09:57:06.299212+0000","last_undegraded":"2023-12-12T09:57:06.299212+0000","last_fullsized":"2023-12-12T09:57:06.299212+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:54:08.677081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497069+0000","last_change":"2023-12-12T09:56:38.277180+0000","last_active":"2023-12-12T09:56:42.497069+0000","last_peered":"2023-12-12T09:56:42.497069+0000","last_clean":"2023-12-12T09:56:42.497069+0000","last_became_active":"2023-12-12T09:56:38.276878+0000","last_became_peered":"2023-12-12T09:56:38.276878+0000","last_unstale":"2023-12-12T09:56:42.497069+0000","last_undegraded":"2023-12-12T09:56:42.497069+0000","last_fullsized":"2023-12-12T09:56:42.497069+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:18:49.704820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682061+0000","last_change":"2023-12-12T09:56:42.512516+0000","last_active":"2023-12-12T09:56:43.682061+0000","last_peered":"2023-12-12T09:56:43.682061+0000","last_clean":"2023-12-12T09:56:43.682061+0000","last_became_active":"2023-12-12T09:56:42.511682+0000","last_became_peered":"2023-12-12T09:56:42.511682+0000","last_unstale":"2023-12-12T09:56:43.682061+0000","last_undegraded":"2023-12-12T09:56:43.682061+0000","last_fullsized":"2023-12-12T09:56:43.682061+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:34:02.287563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.297091+0000","last_change":"2023-12-12T09:56:37.468329+0000","last_active":"2023-12-12T09:57:05.297091+0000","last_peered":"2023-12-12T09:57:05.297091+0000","last_clean":"2023-12-12T09:57:05.297091+0000","last_became_active":"2023-12-12T09:56:37.467886+0000","last_became_peered":"2023-12-12T09:56:37.467886+0000","last_unstale":"2023-12-12T09:57:05.297091+0000","last_undegraded":"2023-12-12T09:57:05.297091+0000","last_fullsized":"2023-12-12T09:57:05.297091+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:23:35.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495836+0000","last_change":"2023-12-12T09:56:38.281280+0000","last_active":"2023-12-12T09:56:42.495836+0000","last_peered":"2023-12-12T09:56:42.495836+0000","last_clean":"2023-12-12T09:56:42.495836+0000","last_became_active":"2023-12-12T09:56:38.280993+0000","last_became_peered":"2023-12-12T09:56:38.280993+0000","last_unstale":"2023-12-12T09:56:42.495836+0000","last_undegraded":"2023-12-12T09:56:42.495836+0000","last_fullsized":"2023-12-12T09:56:42.495836+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:53:44.562981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681331+0000","last_change":"2023-12-12T09:56:42.508920+0000","last_active":"2023-12-12T09:56:43.681331+0000","last_peered":"2023-12-12T09:56:43.681331+0000","last_clean":"2023-12-12T09:56:43.681331+0000","last_became_active":"2023-12-12T09:56:42.508491+0000","last_became_peered":"2023-12-12T09:56:42.508491+0000","last_unstale":"2023-12-12T09:56:43.681331+0000","last_undegraded":"2023-12-12T09:56:43.681331+0000","last_fullsized":"2023-12-12T09:56:43.681331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:12:50.596137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.300318+0000","last_change":"2023-12-12T09:56:37.474701+0000","last_active":"2023-12-12T09:57:06.300318+0000","last_peered":"2023-12-12T09:57:06.300318+0000","last_clean":"2023-12-12T09:57:06.300318+0000","last_became_active":"2023-12-12T09:56:37.473093+0000","last_became_peered":"2023-12-12T09:56:37.473093+0000","last_unstale":"2023-12-12T09:57:06.300318+0000","last_undegraded":"2023-12-12T09:57:06.300318+0000","last_fullsized":"2023-12-12T09:57:06.300318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:42:10.274607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.488056+0000","last_change":"2023-12-12T09:56:38.279775+0000","last_active":"2023-12-12T09:56:42.488056+0000","last_peered":"2023-12-12T09:56:42.488056+0000","last_clean":"2023-12-12T09:56:42.488056+0000","last_became_active":"2023-12-12T09:56:38.278612+0000","last_became_peered":"2023-12-12T09:56:38.278612+0000","last_unstale":"2023-12-12T09:56:42.488056+0000","last_undegraded":"2023-12-12T09:56:42.488056+0000","last_fullsized":"2023-12-12T09:56:42.488056+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:53:57.915759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681301+0000","last_change":"2023-12-12T09:56:42.499828+0000","last_active":"2023-12-12T09:56:43.681301+0000","last_peered":"2023-12-12T09:56:43.681301+0000","last_clean":"2023-12-12T09:56:43.681301+0000","last_became_active":"2023-12-12T09:56:42.499453+0000","last_became_peered":"2023-12-12T09:56:42.499453+0000","last_unstale":"2023-12-12T09:56:43.681301+0000","last_undegraded":"2023-12-12T09:56:43.681301+0000","last_fullsized":"2023-12-12T09:56:43.681301+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:20:12.609145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.785179+0000","last_change":"2023-12-12T09:56:37.475428+0000","last_active":"2023-12-12T09:57:05.785179+0000","last_peered":"2023-12-12T09:57:05.785179+0000","last_clean":"2023-12-12T09:57:05.785179+0000","last_became_active":"2023-12-12T09:56:37.473070+0000","last_became_peered":"2023-12-12T09:56:37.473070+0000","last_unstale":"2023-12-12T09:57:05.785179+0000","last_undegraded":"2023-12-12T09:57:05.785179+0000","last_fullsized":"2023-12-12T09:57:05.785179+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:04:52.292339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.492247+0000","last_change":"2023-12-12T09:56:38.281060+0000","last_active":"2023-12-12T09:56:42.492247+0000","last_peered":"2023-12-12T09:56:42.492247+0000","last_clean":"2023-12-12T09:56:42.492247+0000","last_became_active":"2023-12-12T09:56:38.280725+0000","last_became_peered":"2023-12-12T09:56:38.280725+0000","last_unstale":"2023-12-12T09:56:42.492247+0000","last_undegraded":"2023-12-12T09:56:42.492247+0000","last_fullsized":"2023-12-12T09:56:42.492247+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:14.592213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.262731+0000","last_change":"2023-12-12T09:56:42.578104+0000","last_active":"2023-12-12T09:56:43.262731+0000","last_peered":"2023-12-12T09:56:43.262731+0000","last_clean":"2023-12-12T09:56:43.262731+0000","last_became_active":"2023-12-12T09:56:42.577598+0000","last_became_peered":"2023-12-12T09:56:42.577598+0000","last_unstale":"2023-12-12T09:56:43.262731+0000","last_undegraded":"2023-12-12T09:56:43.262731+0000","last_fullsized":"2023-12-12T09:56:43.262731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:27:49.590817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.500272+0000","last_change":"2023-12-12T09:56:37.465071+0000","last_active":"2023-12-12T09:56:42.500272+0000","last_peered":"2023-12-12T09:56:42.500272+0000","last_clean":"2023-12-12T09:56:42.500272+0000","last_became_active":"2023-12-12T09:56:37.464717+0000","last_became_peered":"2023-12-12T09:56:37.464717+0000","last_unstale":"2023-12-12T09:56:42.500272+0000","last_undegraded":"2023-12-12T09:56:42.500272+0000","last_fullsized":"2023-12-12T09:56:42.500272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:04:52.167228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576905+0000","last_change":"2023-12-12T09:56:38.273276+0000","last_active":"2023-12-12T09:56:42.576905+0000","last_peered":"2023-12-12T09:56:42.576905+0000","last_clean":"2023-12-12T09:56:42.576905+0000","last_became_active":"2023-12-12T09:56:38.272966+0000","last_became_peered":"2023-12-12T09:56:38.272966+0000","last_unstale":"2023-12-12T09:56:42.576905+0000","last_undegraded":"2023-12-12T09:56:42.576905+0000","last_fullsized":"2023-12-12T09:56:42.576905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:22.352843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680378+0000","last_change":"2023-12-12T09:56:42.510808+0000","last_active":"2023-12-12T09:56:43.680378+0000","last_peered":"2023-12-12T09:56:43.680378+0000","last_clean":"2023-12-12T09:56:43.680378+0000","last_became_active":"2023-12-12T09:56:42.510408+0000","last_became_peered":"2023-12-12T09:56:42.510408+0000","last_unstale":"2023-12-12T09:56:43.680378+0000","last_undegraded":"2023-12-12T09:56:43.680378+0000","last_fullsized":"2023-12-12T09:56:43.680378+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:53:08.573345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.263065+0000","last_change":"2023-12-12T09:56:37.468863+0000","last_active":"2023-12-12T09:56:43.263065+0000","last_peered":"2023-12-12T09:56:43.263065+0000","last_clean":"2023-12-12T09:56:43.263065+0000","last_became_active":"2023-12-12T09:56:37.468221+0000","last_became_peered":"2023-12-12T09:56:37.468221+0000","last_unstale":"2023-12-12T09:56:43.263065+0000","last_undegraded":"2023-12-12T09:56:43.263065+0000","last_fullsized":"2023-12-12T09:56:43.263065+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:39:17.559717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.576920+0000","last_change":"2023-12-12T09:56:38.277983+0000","last_active":"2023-12-12T09:56:42.576920+0000","last_peered":"2023-12-12T09:56:42.576920+0000","last_clean":"2023-12-12T09:56:42.576920+0000","last_became_active":"2023-12-12T09:56:38.275926+0000","last_became_peered":"2023-12-12T09:56:38.275926+0000","last_unstale":"2023-12-12T09:56:42.576920+0000","last_undegraded":"2023-12-12T09:56:42.576920+0000","last_fullsized":"2023-12-12T09:56:42.576920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:57:51.474651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680842+0000","last_change":"2023-12-12T09:56:42.577662+0000","last_active":"2023-12-12T09:56:43.680842+0000","last_peered":"2023-12-12T09:56:43.680842+0000","last_clean":"2023-12-12T09:56:43.680842+0000","last_became_active":"2023-12-12T09:56:42.577188+0000","last_became_peered":"2023-12-12T09:56:42.577188+0000","last_unstale":"2023-12-12T09:56:43.680842+0000","last_undegraded":"2023-12-12T09:56:43.680842+0000","last_fullsized":"2023-12-12T09:56:43.680842+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:58:35.341894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487726+0000","last_change":"2023-12-12T09:56:38.279537+0000","last_active":"2023-12-12T09:56:42.487726+0000","last_peered":"2023-12-12T09:56:42.487726+0000","last_clean":"2023-12-12T09:56:42.487726+0000","last_became_active":"2023-12-12T09:56:38.278739+0000","last_became_peered":"2023-12-12T09:56:38.278739+0000","last_unstale":"2023-12-12T09:56:42.487726+0000","last_undegraded":"2023-12-12T09:56:42.487726+0000","last_fullsized":"2023-12-12T09:56:42.487726+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:35:05.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'1","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680095+0000","last_change":"2023-12-12T09:56:37.465726+0000","last_active":"2023-12-12T09:56:43.680095+0000","last_peered":"2023-12-12T09:56:43.680095+0000","last_clean":"2023-12-12T09:56:43.680095+0000","last_became_active":"2023-12-12T09:56:37.465438+0000","last_became_peered":"2023-12-12T09:56:37.465438+0000","last_unstale":"2023-12-12T09:56:43.680095+0000","last_undegraded":"2023-12-12T09:56:43.680095+0000","last_fullsized":"2023-12-12T09:56:43.680095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:17:58.356893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.263337+0000","last_change":"2023-12-12T09:56:42.505943+0000","last_active":"2023-12-12T09:56:43.263337+0000","last_peered":"2023-12-12T09:56:43.263337+0000","last_clean":"2023-12-12T09:56:43.263337+0000","last_became_active":"2023-12-12T09:56:42.505581+0000","last_became_peered":"2023-12-12T09:56:42.505581+0000","last_unstale":"2023-12-12T09:56:43.263337+0000","last_undegraded":"2023-12-12T09:56:43.263337+0000","last_fullsized":"2023-12-12T09:56:43.263337+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:19:08.869868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.488597+0000","last_change":"2023-12-12T09:56:38.284895+0000","last_active":"2023-12-12T09:56:42.488597+0000","last_peered":"2023-12-12T09:56:42.488597+0000","last_clean":"2023-12-12T09:56:42.488597+0000","last_became_active":"2023-12-12T09:56:38.284429+0000","last_became_peered":"2023-12-12T09:56:38.284429+0000","last_unstale":"2023-12-12T09:56:42.488597+0000","last_undegraded":"2023-12-12T09:56:42.488597+0000","last_fullsized":"2023-12-12T09:56:42.488597+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:14.170629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.300517+0000","last_change":"2023-12-12T09:56:37.466748+0000","last_active":"2023-12-12T09:57:05.300517+0000","last_peered":"2023-12-12T09:57:05.300517+0000","last_clean":"2023-12-12T09:57:05.300517+0000","last_became_active":"2023-12-12T09:56:37.466459+0000","last_became_peered":"2023-12-12T09:56:37.466459+0000","last_unstale":"2023-12-12T09:57:05.300517+0000","last_undegraded":"2023-12-12T09:57:05.300517+0000","last_fullsized":"2023-12-12T09:57:05.300517+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:21:43.819845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":322,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.495646+0000","last_change":"2023-12-12T09:55:37.275186+0000","last_active":"2023-12-12T09:56:42.495646+0000","last_peered":"2023-12-12T09:56:42.495646+0000","last_clean":"2023-12-12T09:56:42.495646+0000","last_became_active":"2023-12-12T09:55:37.274808+0000","last_became_peered":"2023-12-12T09:55:37.274808+0000","last_unstale":"2023-12-12T09:56:42.495646+0000","last_undegraded":"2023-12-12T09:56:42.495646+0000","last_fullsized":"2023-12-12T09:56:42.495646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:42.502190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:42.502190+0000","last_clean_scrub_stamp":"2023-12-12T09:52:42.502190+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:27:08.942642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681726+0000","last_change":"2023-12-12T09:56:42.578219+0000","last_active":"2023-12-12T09:56:43.681726+0000","last_peered":"2023-12-12T09:56:43.681726+0000","last_clean":"2023-12-12T09:56:43.681726+0000","last_became_active":"2023-12-12T09:56:42.577589+0000","last_became_peered":"2023-12-12T09:56:42.577589+0000","last_unstale":"2023-12-12T09:56:43.681726+0000","last_undegraded":"2023-12-12T09:56:43.681726+0000","last_fullsized":"2023-12-12T09:56:43.681726+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:40:54.162235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499183+0000","last_change":"2023-12-12T09:56:38.274163+0000","last_active":"2023-12-12T09:56:42.499183+0000","last_peered":"2023-12-12T09:56:42.499183+0000","last_clean":"2023-12-12T09:56:42.499183+0000","last_became_active":"2023-12-12T09:56:38.273837+0000","last_became_peered":"2023-12-12T09:56:38.273837+0000","last_unstale":"2023-12-12T09:56:42.499183+0000","last_undegraded":"2023-12-12T09:56:42.499183+0000","last_fullsized":"2023-12-12T09:56:42.499183+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:46:53.550289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:04.286740+0000","last_change":"2023-12-12T09:56:37.473275+0000","last_active":"2023-12-12T09:57:04.286740+0000","last_peered":"2023-12-12T09:57:04.286740+0000","last_clean":"2023-12-12T09:57:04.286740+0000","last_became_active":"2023-12-12T09:56:37.472973+0000","last_became_peered":"2023-12-12T09:56:37.472973+0000","last_unstale":"2023-12-12T09:57:04.286740+0000","last_undegraded":"2023-12-12T09:57:04.286740+0000","last_fullsized":"2023-12-12T09:57:04.286740+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:03.660259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680813+0000","last_change":"2023-12-12T09:56:42.503299+0000","last_active":"2023-12-12T09:56:43.680813+0000","last_peered":"2023-12-12T09:56:43.680813+0000","last_clean":"2023-12-12T09:56:43.680813+0000","last_became_active":"2023-12-12T09:56:42.502862+0000","last_became_peered":"2023-12-12T09:56:42.502862+0000","last_unstale":"2023-12-12T09:56:43.680813+0000","last_undegraded":"2023-12-12T09:56:43.680813+0000","last_fullsized":"2023-12-12T09:56:43.680813+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:47:14.211916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.301281+0000","last_change":"2023-12-12T09:56:37.476084+0000","last_active":"2023-12-12T09:57:06.301281+0000","last_peered":"2023-12-12T09:57:06.301281+0000","last_clean":"2023-12-12T09:57:06.301281+0000","last_became_active":"2023-12-12T09:56:37.475760+0000","last_became_peered":"2023-12-12T09:56:37.475760+0000","last_unstale":"2023-12-12T09:57:06.301281+0000","last_undegraded":"2023-12-12T09:57:06.301281+0000","last_fullsized":"2023-12-12T09:57:06.301281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:30:43.487809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499907+0000","last_change":"2023-12-12T09:56:38.278502+0000","last_active":"2023-12-12T09:56:42.499907+0000","last_peered":"2023-12-12T09:56:42.499907+0000","last_clean":"2023-12-12T09:56:42.499907+0000","last_became_active":"2023-12-12T09:56:38.278210+0000","last_became_peered":"2023-12-12T09:56:38.278210+0000","last_unstale":"2023-12-12T09:56:42.499907+0000","last_undegraded":"2023-12-12T09:56:42.499907+0000","last_fullsized":"2023-12-12T09:56:42.499907+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:25:09.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680508+0000","last_change":"2023-12-12T09:56:42.577598+0000","last_active":"2023-12-12T09:56:43.680508+0000","last_peered":"2023-12-12T09:56:43.680508+0000","last_clean":"2023-12-12T09:56:43.680508+0000","last_became_active":"2023-12-12T09:56:42.577147+0000","last_became_peered":"2023-12-12T09:56:42.577147+0000","last_unstale":"2023-12-12T09:56:43.680508+0000","last_undegraded":"2023-12-12T09:56:43.680508+0000","last_fullsized":"2023-12-12T09:56:43.680508+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:05:11.279871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'3","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:04.287878+0000","last_change":"2023-12-12T09:56:37.474862+0000","last_active":"2023-12-12T09:57:04.287878+0000","last_peered":"2023-12-12T09:57:04.287878+0000","last_clean":"2023-12-12T09:57:04.287878+0000","last_became_active":"2023-12-12T09:56:37.474463+0000","last_became_peered":"2023-12-12T09:56:37.474463+0000","last_unstale":"2023-12-12T09:57:04.287878+0000","last_undegraded":"2023-12-12T09:57:04.287878+0000","last_fullsized":"2023-12-12T09:57:04.287878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:06.926221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487901+0000","last_change":"2023-12-12T09:56:38.279729+0000","last_active":"2023-12-12T09:56:42.487901+0000","last_peered":"2023-12-12T09:56:42.487901+0000","last_clean":"2023-12-12T09:56:42.487901+0000","last_became_active":"2023-12-12T09:56:38.277634+0000","last_became_peered":"2023-12-12T09:56:38.277634+0000","last_unstale":"2023-12-12T09:56:42.487901+0000","last_undegraded":"2023-12-12T09:56:42.487901+0000","last_fullsized":"2023-12-12T09:56:42.487901+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:06:22.594507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.554701+0000","last_change":"2023-12-12T09:56:42.502342+0000","last_active":"2023-12-12T09:56:43.554701+0000","last_peered":"2023-12-12T09:56:43.554701+0000","last_clean":"2023-12-12T09:56:43.554701+0000","last_became_active":"2023-12-12T09:56:42.501313+0000","last_became_peered":"2023-12-12T09:56:42.501313+0000","last_unstale":"2023-12-12T09:56:43.554701+0000","last_undegraded":"2023-12-12T09:56:43.554701+0000","last_fullsized":"2023-12-12T09:56:43.554701+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:12:28.144029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.299191+0000","last_change":"2023-12-12T09:56:37.464428+0000","last_active":"2023-12-12T09:57:06.299191+0000","last_peered":"2023-12-12T09:57:06.299191+0000","last_clean":"2023-12-12T09:57:06.299191+0000","last_became_active":"2023-12-12T09:56:37.464159+0000","last_became_peered":"2023-12-12T09:56:37.464159+0000","last_unstale":"2023-12-12T09:57:06.299191+0000","last_undegraded":"2023-12-12T09:57:06.299191+0000","last_fullsized":"2023-12-12T09:57:06.299191+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:14:53.779852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496677+0000","last_change":"2023-12-12T09:56:38.277023+0000","last_active":"2023-12-12T09:56:42.496677+0000","last_peered":"2023-12-12T09:56:42.496677+0000","last_clean":"2023-12-12T09:56:42.496677+0000","last_became_active":"2023-12-12T09:56:38.276708+0000","last_became_peered":"2023-12-12T09:56:38.276708+0000","last_unstale":"2023-12-12T09:56:42.496677+0000","last_undegraded":"2023-12-12T09:56:42.496677+0000","last_fullsized":"2023-12-12T09:56:42.496677+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:08:28.244491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681673+0000","last_change":"2023-12-12T09:56:42.578025+0000","last_active":"2023-12-12T09:56:43.681673+0000","last_peered":"2023-12-12T09:56:43.681673+0000","last_clean":"2023-12-12T09:56:43.681673+0000","last_became_active":"2023-12-12T09:56:42.577570+0000","last_became_peered":"2023-12-12T09:56:42.577570+0000","last_unstale":"2023-12-12T09:56:43.681673+0000","last_undegraded":"2023-12-12T09:56:43.681673+0000","last_fullsized":"2023-12-12T09:56:43.681673+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:51:02.998355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.276026+0000","last_change":"2023-12-12T09:56:37.467815+0000","last_active":"2023-12-12T09:57:03.276026+0000","last_peered":"2023-12-12T09:57:03.276026+0000","last_clean":"2023-12-12T09:57:03.276026+0000","last_became_active":"2023-12-12T09:56:37.467110+0000","last_became_peered":"2023-12-12T09:56:37.467110+0000","last_unstale":"2023-12-12T09:57:03.276026+0000","last_undegraded":"2023-12-12T09:57:03.276026+0000","last_fullsized":"2023-12-12T09:57:03.276026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:40:24.325774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.491987+0000","last_change":"2023-12-12T09:56:38.284770+0000","last_active":"2023-12-12T09:56:42.491987+0000","last_peered":"2023-12-12T09:56:42.491987+0000","last_clean":"2023-12-12T09:56:42.491987+0000","last_became_active":"2023-12-12T09:56:38.284459+0000","last_became_peered":"2023-12-12T09:56:38.284459+0000","last_unstale":"2023-12-12T09:56:42.491987+0000","last_undegraded":"2023-12-12T09:56:42.491987+0000","last_fullsized":"2023-12-12T09:56:42.491987+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:02:42.787163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.682121+0000","last_change":"2023-12-12T09:56:42.512278+0000","last_active":"2023-12-12T09:56:43.682121+0000","last_peered":"2023-12-12T09:56:43.682121+0000","last_clean":"2023-12-12T09:56:43.682121+0000","last_became_active":"2023-12-12T09:56:42.511598+0000","last_became_peered":"2023-12-12T09:56:42.511598+0000","last_unstale":"2023-12-12T09:56:43.682121+0000","last_undegraded":"2023-12-12T09:56:43.682121+0000","last_fullsized":"2023-12-12T09:56:43.682121+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:00:03.571614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.298234+0000","last_change":"2023-12-12T09:56:37.469274+0000","last_active":"2023-12-12T09:57:05.298234+0000","last_peered":"2023-12-12T09:57:05.298234+0000","last_clean":"2023-12-12T09:57:05.298234+0000","last_became_active":"2023-12-12T09:56:37.468947+0000","last_became_peered":"2023-12-12T09:56:37.468947+0000","last_unstale":"2023-12-12T09:57:05.298234+0000","last_undegraded":"2023-12-12T09:57:05.298234+0000","last_fullsized":"2023-12-12T09:57:05.298234+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:26:25.559385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.491775+0000","last_change":"2023-12-12T09:56:38.274415+0000","last_active":"2023-12-12T09:56:42.491775+0000","last_peered":"2023-12-12T09:56:42.491775+0000","last_clean":"2023-12-12T09:56:42.491775+0000","last_became_active":"2023-12-12T09:56:38.274119+0000","last_became_peered":"2023-12-12T09:56:38.274119+0000","last_unstale":"2023-12-12T09:56:42.491775+0000","last_undegraded":"2023-12-12T09:56:42.491775+0000","last_fullsized":"2023-12-12T09:56:42.491775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:50.160355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681718+0000","last_change":"2023-12-12T09:56:42.506811+0000","last_active":"2023-12-12T09:56:43.681718+0000","last_peered":"2023-12-12T09:56:43.681718+0000","last_clean":"2023-12-12T09:56:43.681718+0000","last_became_active":"2023-12-12T09:56:42.506374+0000","last_became_peered":"2023-12-12T09:56:42.506374+0000","last_unstale":"2023-12-12T09:56:43.681718+0000","last_undegraded":"2023-12-12T09:56:43.681718+0000","last_fullsized":"2023-12-12T09:56:43.681718+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:50:54.647403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.218120+0000","last_change":"2023-12-12T09:56:37.473548+0000","last_active":"2023-12-12T09:57:05.218120+0000","last_peered":"2023-12-12T09:57:05.218120+0000","last_clean":"2023-12-12T09:57:05.218120+0000","last_became_active":"2023-12-12T09:56:37.473240+0000","last_became_peered":"2023-12-12T09:56:37.473240+0000","last_unstale":"2023-12-12T09:57:05.218120+0000","last_undegraded":"2023-12-12T09:57:05.218120+0000","last_fullsized":"2023-12-12T09:57:05.218120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:42:22.921437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487444+0000","last_change":"2023-12-12T09:56:38.276853+0000","last_active":"2023-12-12T09:56:42.487444+0000","last_peered":"2023-12-12T09:56:42.487444+0000","last_clean":"2023-12-12T09:56:42.487444+0000","last_became_active":"2023-12-12T09:56:38.276567+0000","last_became_peered":"2023-12-12T09:56:38.276567+0000","last_unstale":"2023-12-12T09:56:42.487444+0000","last_undegraded":"2023-12-12T09:56:42.487444+0000","last_fullsized":"2023-12-12T09:56:42.487444+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:21:15.069389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.681616+0000","last_change":"2023-12-12T09:56:42.579135+0000","last_active":"2023-12-12T09:56:43.681616+0000","last_peered":"2023-12-12T09:56:43.681616+0000","last_clean":"2023-12-12T09:56:43.681616+0000","last_became_active":"2023-12-12T09:56:42.578513+0000","last_became_peered":"2023-12-12T09:56:42.578513+0000","last_unstale":"2023-12-12T09:56:43.681616+0000","last_undegraded":"2023-12-12T09:56:43.681616+0000","last_fullsized":"2023-12-12T09:56:43.681616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:36:27.337524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:03.276148+0000","last_change":"2023-12-12T09:56:37.475479+0000","last_active":"2023-12-12T09:57:03.276148+0000","last_peered":"2023-12-12T09:57:03.276148+0000","last_clean":"2023-12-12T09:57:03.276148+0000","last_became_active":"2023-12-12T09:56:37.473657+0000","last_became_peered":"2023-12-12T09:56:37.473657+0000","last_unstale":"2023-12-12T09:57:03.276148+0000","last_undegraded":"2023-12-12T09:57:03.276148+0000","last_fullsized":"2023-12-12T09:57:03.276148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:49:07.610160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496445+0000","last_change":"2023-12-12T09:56:38.279661+0000","last_active":"2023-12-12T09:56:42.496445+0000","last_peered":"2023-12-12T09:56:42.496445+0000","last_clean":"2023-12-12T09:56:42.496445+0000","last_became_active":"2023-12-12T09:56:38.279330+0000","last_became_peered":"2023-12-12T09:56:38.279330+0000","last_unstale":"2023-12-12T09:56:42.496445+0000","last_undegraded":"2023-12-12T09:56:42.496445+0000","last_fullsized":"2023-12-12T09:56:42.496445+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:19:15.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680487+0000","last_change":"2023-12-12T09:56:42.502911+0000","last_active":"2023-12-12T09:56:43.680487+0000","last_peered":"2023-12-12T09:56:43.680487+0000","last_clean":"2023-12-12T09:56:43.680487+0000","last_became_active":"2023-12-12T09:56:42.502467+0000","last_became_peered":"2023-12-12T09:56:42.502467+0000","last_unstale":"2023-12-12T09:56:43.680487+0000","last_undegraded":"2023-12-12T09:56:43.680487+0000","last_fullsized":"2023-12-12T09:56:43.680487+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:45:42.568227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:05.296154+0000","last_change":"2023-12-12T09:56:37.466958+0000","last_active":"2023-12-12T09:57:05.296154+0000","last_peered":"2023-12-12T09:57:05.296154+0000","last_clean":"2023-12-12T09:57:05.296154+0000","last_became_active":"2023-12-12T09:56:37.466591+0000","last_became_peered":"2023-12-12T09:56:37.466591+0000","last_unstale":"2023-12-12T09:57:05.296154+0000","last_undegraded":"2023-12-12T09:57:05.296154+0000","last_fullsized":"2023-12-12T09:57:05.296154+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:25:35.605587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497525+0000","last_change":"2023-12-12T09:56:38.279142+0000","last_active":"2023-12-12T09:56:42.497525+0000","last_peered":"2023-12-12T09:56:42.497525+0000","last_clean":"2023-12-12T09:56:42.497525+0000","last_became_active":"2023-12-12T09:56:38.278441+0000","last_became_peered":"2023-12-12T09:56:38.278441+0000","last_unstale":"2023-12-12T09:56:42.497525+0000","last_undegraded":"2023-12-12T09:56:42.497525+0000","last_fullsized":"2023-12-12T09:56:42.497525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:36:49.024628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.261911+0000","last_change":"2023-12-12T09:56:42.506159+0000","last_active":"2023-12-12T09:56:43.261911+0000","last_peered":"2023-12-12T09:56:43.261911+0000","last_clean":"2023-12-12T09:56:43.261911+0000","last_became_active":"2023-12-12T09:56:42.505704+0000","last_became_peered":"2023-12-12T09:56:42.505704+0000","last_unstale":"2023-12-12T09:56:43.261911+0000","last_undegraded":"2023-12-12T09:56:43.261911+0000","last_fullsized":"2023-12-12T09:56:43.261911+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:51:55.633237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:57:06.300168+0000","last_change":"2023-12-12T09:56:37.470470+0000","last_active":"2023-12-12T09:57:06.300168+0000","last_peered":"2023-12-12T09:57:06.300168+0000","last_clean":"2023-12-12T09:57:06.300168+0000","last_became_active":"2023-12-12T09:56:37.469991+0000","last_became_peered":"2023-12-12T09:56:37.469991+0000","last_unstale":"2023-12-12T09:57:06.300168+0000","last_undegraded":"2023-12-12T09:57:06.300168+0000","last_fullsized":"2023-12-12T09:57:06.300168+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:49.239075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496992+0000","last_change":"2023-12-12T09:56:38.281338+0000","last_active":"2023-12-12T09:56:42.496992+0000","last_peered":"2023-12-12T09:56:42.496992+0000","last_clean":"2023-12-12T09:56:42.496992+0000","last_became_active":"2023-12-12T09:56:38.281048+0000","last_became_peered":"2023-12-12T09:56:38.281048+0000","last_unstale":"2023-12-12T09:56:42.496992+0000","last_undegraded":"2023-12-12T09:56:42.496992+0000","last_fullsized":"2023-12-12T09:56:42.496992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:27:27.940574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-12-12T09:56:43.680622+0000","last_change":"2023-12-12T09:56:42.500612+0000","last_active":"2023-12-12T09:56:43.680622+0000","last_peered":"2023-12-12T09:56:43.680622+0000","last_clean":"2023-12-12T09:56:43.680622+0000","last_became_active":"2023-12-12T09:56:42.500202+0000","last_became_peered":"2023-12-12T09:56:42.500202+0000","last_unstale":"2023-12-12T09:56:43.680622+0000","last_undegraded":"2023-12-12T09:56:43.680622+0000","last_fullsized":"2023-12-12T09:56:43.680622+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:11:07.397950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499211+0000","last_change":"2023-12-12T09:56:37.465748+0000","last_active":"2023-12-12T09:56:42.499211+0000","last_peered":"2023-12-12T09:56:42.499211+0000","last_clean":"2023-12-12T09:56:42.499211+0000","last_became_active":"2023-12-12T09:56:37.465408+0000","last_became_peered":"2023-12-12T09:56:37.465408+0000","last_unstale":"2023-12-12T09:56:42.499211+0000","last_undegraded":"2023-12-12T09:56:42.499211+0000","last_fullsized":"2023-12-12T09:56:42.499211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:58:38.777008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497525+0000","last_change":"2023-12-12T09:56:38.283934+0000","last_active":"2023-12-12T09:56:42.497525+0000","last_peered":"2023-12-12T09:56:42.497525+0000","last_clean":"2023-12-12T09:56:42.497525+0000","last_became_active":"2023-12-12T09:56:38.283605+0000","last_became_peered":"2023-12-12T09:56:38.283605+0000","last_unstale":"2023-12-12T09:56:42.497525+0000","last_undegraded":"2023-12-12T09:56:42.497525+0000","last_fullsized":"2023-12-12T09:56:42.497525+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:52:46.792159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.491862+0000","last_change":"2023-12-12T09:56:38.279834+0000","last_active":"2023-12-12T09:56:42.491862+0000","last_peered":"2023-12-12T09:56:42.491862+0000","last_clean":"2023-12-12T09:56:42.491862+0000","last_became_active":"2023-12-12T09:56:38.279353+0000","last_became_peered":"2023-12-12T09:56:38.279353+0000","last_unstale":"2023-12-12T09:56:42.491862+0000","last_undegraded":"2023-12-12T09:56:42.491862+0000","last_fullsized":"2023-12-12T09:56:42.491862+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:21:52.237458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.497845+0000","last_change":"2023-12-12T09:56:38.279341+0000","last_active":"2023-12-12T09:56:42.497845+0000","last_peered":"2023-12-12T09:56:42.497845+0000","last_clean":"2023-12-12T09:56:42.497845+0000","last_became_active":"2023-12-12T09:56:38.279054+0000","last_became_peered":"2023-12-12T09:56:38.279054+0000","last_unstale":"2023-12-12T09:56:42.497845+0000","last_undegraded":"2023-12-12T09:56:42.497845+0000","last_fullsized":"2023-12-12T09:56:42.497845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:55:33.729689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496023+0000","last_change":"2023-12-12T09:56:38.277052+0000","last_active":"2023-12-12T09:56:42.496023+0000","last_peered":"2023-12-12T09:56:42.496023+0000","last_clean":"2023-12-12T09:56:42.496023+0000","last_became_active":"2023-12-12T09:56:38.276236+0000","last_became_peered":"2023-12-12T09:56:38.276236+0000","last_unstale":"2023-12-12T09:56:42.496023+0000","last_undegraded":"2023-12-12T09:56:42.496023+0000","last_fullsized":"2023-12-12T09:56:42.496023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:31:52.066130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.494775+0000","last_change":"2023-12-12T09:56:38.280936+0000","last_active":"2023-12-12T09:56:42.494775+0000","last_peered":"2023-12-12T09:56:42.494775+0000","last_clean":"2023-12-12T09:56:42.494775+0000","last_became_active":"2023-12-12T09:56:38.280600+0000","last_became_peered":"2023-12-12T09:56:38.280600+0000","last_unstale":"2023-12-12T09:56:42.494775+0000","last_undegraded":"2023-12-12T09:56:42.494775+0000","last_fullsized":"2023-12-12T09:56:42.494775+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:40:36.636954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499777+0000","last_change":"2023-12-12T09:56:38.278839+0000","last_active":"2023-12-12T09:56:42.499777+0000","last_peered":"2023-12-12T09:56:42.499777+0000","last_clean":"2023-12-12T09:56:42.499777+0000","last_became_active":"2023-12-12T09:56:38.276299+0000","last_became_peered":"2023-12-12T09:56:38.276299+0000","last_unstale":"2023-12-12T09:56:42.499777+0000","last_undegraded":"2023-12-12T09:56:42.499777+0000","last_fullsized":"2023-12-12T09:56:42.499777+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:38:26.830037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.492126+0000","last_change":"2023-12-12T09:56:38.282200+0000","last_active":"2023-12-12T09:56:42.492126+0000","last_peered":"2023-12-12T09:56:42.492126+0000","last_clean":"2023-12-12T09:56:42.492126+0000","last_became_active":"2023-12-12T09:56:38.281938+0000","last_became_peered":"2023-12-12T09:56:38.281938+0000","last_unstale":"2023-12-12T09:56:42.492126+0000","last_undegraded":"2023-12-12T09:56:42.492126+0000","last_fullsized":"2023-12-12T09:56:42.492126+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:45:14.391963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496626+0000","last_change":"2023-12-12T09:56:38.276857+0000","last_active":"2023-12-12T09:56:42.496626+0000","last_peered":"2023-12-12T09:56:42.496626+0000","last_clean":"2023-12-12T09:56:42.496626+0000","last_became_active":"2023-12-12T09:56:38.276562+0000","last_became_peered":"2023-12-12T09:56:38.276562+0000","last_unstale":"2023-12-12T09:56:42.496626+0000","last_undegraded":"2023-12-12T09:56:42.496626+0000","last_fullsized":"2023-12-12T09:56:42.496626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:54:28.262613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.488506+0000","last_change":"2023-12-12T09:56:38.278006+0000","last_active":"2023-12-12T09:56:42.488506+0000","last_peered":"2023-12-12T09:56:42.488506+0000","last_clean":"2023-12-12T09:56:42.488506+0000","last_became_active":"2023-12-12T09:56:38.277222+0000","last_became_peered":"2023-12-12T09:56:38.277222+0000","last_unstale":"2023-12-12T09:56:42.488506+0000","last_undegraded":"2023-12-12T09:56:42.488506+0000","last_fullsized":"2023-12-12T09:56:42.488506+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:46:50.602603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.487631+0000","last_change":"2023-12-12T09:56:38.282872+0000","last_active":"2023-12-12T09:56:42.487631+0000","last_peered":"2023-12-12T09:56:42.487631+0000","last_clean":"2023-12-12T09:56:42.487631+0000","last_became_active":"2023-12-12T09:56:38.282582+0000","last_became_peered":"2023-12-12T09:56:38.282582+0000","last_unstale":"2023-12-12T09:56:42.487631+0000","last_undegraded":"2023-12-12T09:56:42.487631+0000","last_fullsized":"2023-12-12T09:56:42.487631+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:36:35.447323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.486961+0000","last_change":"2023-12-12T09:56:38.286707+0000","last_active":"2023-12-12T09:56:42.486961+0000","last_peered":"2023-12-12T09:56:42.486961+0000","last_clean":"2023-12-12T09:56:42.486961+0000","last_became_active":"2023-12-12T09:56:38.286382+0000","last_became_peered":"2023-12-12T09:56:38.286382+0000","last_unstale":"2023-12-12T09:56:42.486961+0000","last_undegraded":"2023-12-12T09:56:42.486961+0000","last_fullsized":"2023-12-12T09:56:42.486961+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:10:12.412091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.496230+0000","last_change":"2023-12-12T09:56:38.276461+0000","last_active":"2023-12-12T09:56:42.496230+0000","last_peered":"2023-12-12T09:56:42.496230+0000","last_clean":"2023-12-12T09:56:42.496230+0000","last_became_active":"2023-12-12T09:56:38.276173+0000","last_became_peered":"2023-12-12T09:56:38.276173+0000","last_unstale":"2023-12-12T09:56:42.496230+0000","last_undegraded":"2023-12-12T09:56:42.496230+0000","last_fullsized":"2023-12-12T09:56:42.496230+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:03:53.874132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.499853+0000","last_change":"2023-12-12T09:56:38.276993+0000","last_active":"2023-12-12T09:56:42.499853+0000","last_peered":"2023-12-12T09:56:42.499853+0000","last_clean":"2023-12-12T09:56:42.499853+0000","last_became_active":"2023-12-12T09:56:38.276629+0000","last_became_peered":"2023-12-12T09:56:38.276629+0000","last_unstale":"2023-12-12T09:56:42.499853+0000","last_undegraded":"2023-12-12T09:56:42.499853+0000","last_fullsized":"2023-12-12T09:56:42.499853+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:17:39.363638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-12T09:56:42.493210+0000","last_change":"2023-12-12T09:56:38.281874+0000","last_active":"2023-12-12T09:56:42.493210+0000","last_peered":"2023-12-12T09:56:42.493210+0000","last_clean":"2023-12-12T09:56:42.493210+0000","last_became_active":"2023-12-12T09:56:38.281563+0000","last_became_peered":"2023-12-12T09:56:38.281563+0000","last_unstale":"2023-12-12T09:56:42.493210+0000","last_undegraded":"2023-12-12T09:56:42.493210+0000","last_fullsized":"2023-12-12T09:56:42.493210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:32:18.886026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":10158,"num_objects":77,"num_object_clones":0,"num_object_copies":231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":16,"num_read_kb":15,"num_write":163,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":50,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":356352,"data_stored":72011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449104,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":962560,"data_stored":467691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.57799999999999996}]},{"osd":1,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.75},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.88200000000000001}]},{"osd":2,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.77500000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.86199999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.97799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":9,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90500000000000003}]},{"osd":10,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.92600000000000005}]}]},{"osd":10,"up_from":70,"seq":300647710740,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1061942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.71299999999999997}]},{"osd":2,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.753},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72699999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.266,"5min":1.266,"15min":1.266},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68500000000000005}]},{"osd":4,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.53400000000000003}]},{"osd":5,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1,"5min":1,"15min":1},"last":0.78600000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.628},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.80700000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.48199999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.878}]},{"osd":9,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.4370000000000001,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.81999999999999995}]},{"osd":11,"last update":"Tue Dec 12 09:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.84399999999999997}]}]},{"osd":9,"up_from":65,"seq":279172874265,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":467927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53300000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.437}]},{"osd":3,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.621},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.69699999999999995}]},{"osd":5,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.504}]},{"osd":6,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.179},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.66000000000000003}]},{"osd":8,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.49299999999999999}]},{"osd":10,"last update":"Tue Dec 12 09:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.73099999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":8,"up_from":57,"seq":244813135902,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":472705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.44600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.78800000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.436},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.307,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.59399999999999997}]},{"osd":3,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55200000000000005}]},{"osd":4,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.623},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.74399999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.752,"5min":0.874,"15min":0.874},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70499999999999996},"min":{"1min":0.53700000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.65700000000000003}]},{"osd":7,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.73199999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95499999999999996,"5min":1.117,"15min":1.117},"last":0.73099999999999998}]},{"osd":9,"last update":"Tue Dec 12 09:56:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70899999999999996}]},{"osd":10,"last update":"Tue Dec 12 09:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80600000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":7,"up_from":50,"seq":214748364833,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28540,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716612,"statfs":{"total":95995035648,"available":95965810688,"internally_reserved":0,"allocated":1560576,"data_stored":1058665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.311,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50900000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.42399999999999999,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.503},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73899999999999999,"5min":0.746,"15min":0.746},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.84699999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59499999999999997,"15min":0.58699999999999997},"min":{"1min":0.374,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.72799999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.56499999999999995,"15min":0.55800000000000005},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60299999999999998,"15min":0.59199999999999997},"min":{"1min":0.45600000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.871,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.35499999999999998,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83399999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53900000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82399999999999995,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55800000000000005}]},{"osd":5,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66200000000000003,"15min":0.66100000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86899999999999999,"5min":1.093,"15min":1.093},"last":0.748},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.47299999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.83199999999999996,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65600000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.68899999999999995,"15min":0.68700000000000006},"min":{"1min":0.497,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93000000000000005,"5min":1.113,"15min":1.113},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.94499999999999995,"5min":1.018,"15min":1.018},"last":0.64200000000000002}]},{"osd":8,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54100000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81899999999999995}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83499999999999996}]},{"osd":10,"last update":"Tue Dec 12 09:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.80500000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528357,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":467109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.39000000000000001,"15min":0.38},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.42999999999999999,"15min":0.42299999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.434}]},{"osd":1,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.56699999999999995,"15min":0.56000000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.60899999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57599999999999996,"15min":0.56999999999999995},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.58699999999999997}]},{"osd":3,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.48699999999999999,"15min":0.47499999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.498},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.32200000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.44600000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64300000000000002,"15min":0.63600000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.73599999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.67200000000000004,"15min":0.66600000000000004},"min":{"1min":0.54500000000000004,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.68200000000000005,"15min":0.66600000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.86299999999999999,"15min":0.873},"min":{"1min":0.51200000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.091,"5min":5.069,"15min":5.069},"last":0.72499999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.745,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.56799999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.63300000000000001}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.51600000000000001}]},{"osd":10,"last update":"Tue Dec 12 09:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.751}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":471531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.51700000000000002,"15min":0.495},"min":{"1min":0.377,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.51600000000000001,"15min":0.48699999999999999},"min":{"1min":0.31900000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.55900000000000005}]},{"osd":2,"last update":"Tue Dec 12 09:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.59599999999999997,"15min":0.54700000000000004},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":1.032,"15min":1.032},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91400000000000003,"5min":1.079,"15min":1.079},"last":0.85199999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.55100000000000005,"15min":0.52500000000000002},"min":{"1min":0.34399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.57499999999999996,"15min":0.54300000000000004},"min":{"1min":0.46100000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.751}]},{"osd":4,"last update":"Tue Dec 12 09:56:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.41499999999999998,"15min":0.38200000000000001},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.495,"5min":0.42099999999999999,"15min":0.40000000000000002},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58599999999999997,"15min":0.55500000000000005},"min":{"1min":0.40899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":1.018,"15min":1.018},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.59999999999999998,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.58199999999999996,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.374,"15min":0.374},"max":{"1min":0.80000000000000004,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.61899999999999999,"15min":0.60199999999999998},"min":{"1min":0.44700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89300000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.80400000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36099999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":1.042},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.58699999999999997,"15min":0.56899999999999995},"min":{"1min":0.41299999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":1.012}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.48699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.98799999999999999}]},{"osd":9,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.86799999999999999}]},{"osd":10,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.0269999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":468587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.51900000000000002,"15min":0.48699999999999999},"min":{"1min":0.38900000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.75800000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.52900000000000003,"15min":0.47499999999999998},"min":{"1min":0.40999999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.61199999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.56399999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77400000000000002,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.55900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.89300000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:55:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.41399999999999998,"15min":0.39500000000000002},"min":{"1min":0.35899999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.44900000000000001,"15min":0.436},"min":{"1min":0.35999999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55200000000000005,"15min":0.53000000000000003},"min":{"1min":0.38600000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.60499999999999998,"15min":0.58499999999999996},"min":{"1min":0.47199999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.67300000000000004}]},{"osd":5,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.63400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.80300000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:56:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.54200000000000004,"15min":0.53700000000000003},"min":{"1min":0.35199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":1.016,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.87}]},{"osd":7,"last update":"Tue Dec 12 09:55:34 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.90800000000000003},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.95999999999999996}]},{"osd":8,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":1.0149999999999999}]},{"osd":9,"last update":"Tue Dec 12 09:56:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.98999999999999999}]},{"osd":10,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":1.0329999999999999}]},{"osd":11,"last update":"Tue Dec 12 09:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.0820000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":472590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.34999999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.79200000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.505},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.502,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.754,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.72699999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.623}]},{"osd":3,"last update":"Tue Dec 12 09:56:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.56999999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.47399999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.75,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.66900000000000004}]},{"osd":4,"last update":"Tue Dec 12 09:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.54900000000000004,"15min":0.51300000000000001},"min":{"1min":0.47099999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.82599999999999996}]},{"osd":5,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45000000000000001,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.46800000000000003,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.52500000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.60799999999999998,"15min":0.59499999999999997},"min":{"1min":0.46700000000000003,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.79100000000000004,"15min":0.80700000000000005},"min":{"1min":0.45500000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93600000000000005,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.77700000000000002}]},{"osd":7,"last update":"Tue Dec 12 09:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.81399999999999995}]},{"osd":8,"last update":"Tue Dec 12 09:56:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76200000000000001}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.877}]},{"osd":10,"last update":"Tue Dec 12 09:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.747}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182450,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":970752,"data_stored":467062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.36399999999999999,"15min":0.34699999999999998},"min":{"1min":0.33700000000000002,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.37},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.377,"15min":0.35899999999999999},"min":{"1min":0.308,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.31900000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.53700000000000003,"15min":0.51500000000000001},"min":{"1min":0.40799999999999997,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.55400000000000005,"15min":0.53400000000000003},"min":{"1min":0.48099999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.65800000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:56:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.56299999999999994},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.57999999999999996,"15min":0.56499999999999995},"min":{"1min":0.48199999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.80300000000000005}]},{"osd":4,"last update":"Tue Dec 12 09:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.56200000000000006,"15min":0.53200000000000003},"min":{"1min":0.39800000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.41099999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60399999999999998,"15min":0.58099999999999996},"min":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.434,"5min":0.432,"15min":0.432},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.73499999999999999}]},{"osd":6,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.52600000000000002},"min":{"1min":0.42499999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.58599999999999997,"15min":0.56399999999999995},"min":{"1min":0.46300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.75}]},{"osd":7,"last update":"Tue Dec 12 09:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.51200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.69599999999999995,"15min":0.68400000000000005},"min":{"1min":0.62,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.63400000000000001}]},{"osd":8,"last update":"Tue Dec 12 09:55:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58099999999999996}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.84599999999999997}]},{"osd":10,"last update":"Tue Dec 12 09:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]}]},{"osd":4,"up_from":32,"seq":137438953519,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":468197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72899999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.23999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.61499999999999999,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.25800000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.373}]},{"osd":2,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71799999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.53800000000000003,"15min":0.52300000000000002},"min":{"1min":0.38,"5min":0.373,"15min":0.373},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53700000000000003}]},{"osd":3,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.59699999999999998,"15min":0.60499999999999998},"min":{"1min":0.41299999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.82799999999999996,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.60099999999999998},"min":{"1min":0.42299999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79400000000000004,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.51300000000000001}]},{"osd":5,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.495}]},{"osd":6,"last update":"Tue Dec 12 09:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65300000000000002,"15min":0.65200000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.66700000000000004,"15min":0.66600000000000004},"min":{"1min":0.496,"5min":0.442,"15min":0.442},"max":{"1min":0.871,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.59599999999999997}]},{"osd":7,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.54600000000000004,"15min":0.53300000000000003},"min":{"1min":0.40699999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.56699999999999995,"15min":0.55700000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.89100000000000001,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60699999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.53800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92700000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.72199999999999998}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.70699999999999996}]},{"osd":10,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.621},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.65900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]}]},{"osd":5,"up_from":37,"seq":158913789995,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28580,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716572,"statfs":{"total":95995035648,"available":95965769728,"internally_reserved":0,"allocated":1601536,"data_stored":1064781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51900000000000002,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.64800000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.53900000000000003,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.72199999999999998}]},{"osd":2,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.27500000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.39600000000000002},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48099999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95199999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.89000000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.439,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58399999999999996,"15min":0.56299999999999994},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.75900000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:56:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.61099999999999999,"15min":0.59799999999999998},"min":{"1min":0.53100000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85899999999999999,"5min":1.056,"15min":1.056},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.52200000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.53200000000000003}]},{"osd":7,"last update":"Tue Dec 12 09:56:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.496,"5min":0.441,"15min":0.441},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69099999999999995,"15min":0.68999999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.84999999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.432,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.55900000000000005},"min":{"1min":0.375,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58499999999999996}]},{"osd":9,"last update":"Tue Dec 12 09:56:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.79900000000000004}]},{"osd":10,"last update":"Tue Dec 12 09:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.82699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":22971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":32760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:57:08.534 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-12T09:57:08.534 INFO:tasks.cephadm:Setting up client nodes... 2023-12-12T09:57:08.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-12T09:57:08.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:08 smithi173 ceph-mon[103308]: from='client.14904 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:57:08.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:08 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] up:active 2023-12-12T09:57:08.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:08 smithi173 ceph-mon[103308]: fsmap cephfs:5 7 up:standby 5 up:active 2023-12-12T09:57:08.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:08 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T09:57:08.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:08 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2381901367' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:08.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:08 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1696753927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:09.169 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:09.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:09 smithi173 ceph-mon[103308]: pgmap v277: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2023-12-12T09:57:09.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:09 smithi173 ceph-mon[103308]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:09.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:09 smithi173 ceph-mon[103308]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:09.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:09 smithi042 ceph-mon[98894]: pgmap v277: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2023-12-12T09:57:09.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:09 smithi042 ceph-mon[98894]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:09.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:09 smithi042 ceph-mon[98894]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:09.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:09 smithi111 ceph-mon[102565]: pgmap v277: 129 pgs: 129 active+clean; 587 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 3.5 KiB/s wr, 11 op/s 2023-12-12T09:57:09.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:09 smithi111 ceph-mon[102565]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:09.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:09 smithi111 ceph-mon[102565]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.534+0000 7f012c22d700 1 -- 172.21.15.42:0/352784272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01241004f0 msgr2=0x7f0124100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.534+0000 7f012c22d700 1 --2- 172.21.15.42:0/352784272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01241004f0 0x7f0124100910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f01200099c0 tx=0x7f0120009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.534+0000 7f012c22d700 1 -- 172.21.15.42:0/352784272 shutdown_connections 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.534+0000 7f012c22d700 1 --2- 172.21.15.42:0/352784272 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0124102090 0x7f0124108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.534+0000 7f012c22d700 1 --2- 172.21.15.42:0/352784272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01241004f0 0x7f0124100910 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.534+0000 7f012c22d700 1 --2- 172.21.15.42:0/352784272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01241016f0 0x7f0124101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.534+0000 7f012c22d700 1 -- 172.21.15.42:0/352784272 >> 172.21.15.42:0/352784272 conn(0x7f01240fba90 msgr2=0x7f01240fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.534+0000 7f012c22d700 1 -- 172.21.15.42:0/352784272 shutdown_connections 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.534+0000 7f012c22d700 1 -- 172.21.15.42:0/352784272 wait complete. 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012c22d700 1 Processor -- start 2023-12-12T09:57:10.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012c22d700 1 -- start start 2023-12-12T09:57:10.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012c22d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01241004f0 0x7f012406a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:10.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012c22d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01241016f0 0x7f012419c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:10.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012c22d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0124102090 0x7f012419e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:10.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012c22d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012419ef30 con 0x7f0124102090 2023-12-12T09:57:10.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012c22d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012419f0a0 con 0x7f01241004f0 2023-12-12T09:57:10.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012c22d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012419f260 con 0x7f01241016f0 2023-12-12T09:57:10.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012a7ca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0124102090 0x7f012419e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012a7ca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0124102090 0x7f012419e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34036/0 (socket says 172.21.15.42:34036) 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f012a7ca700 1 -- 172.21.15.42:0/2860592319 learned_addr learned my addr 172.21.15.42:0/2860592319 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f01297c8700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01241016f0 0x7f012419c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.535+0000 7f0129fc9700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01241004f0 0x7f012406a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f012a7ca700 1 -- 172.21.15.42:0/2860592319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01241016f0 msgr2=0x7f012419c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f012a7ca700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01241016f0 0x7f012419c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f012a7ca700 1 -- 172.21.15.42:0/2860592319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01241004f0 msgr2=0x7f012406a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f012a7ca700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01241004f0 0x7f012406a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f012a7ca700 1 -- 172.21.15.42:0/2860592319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01200095f0 con 0x7f0124102090 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f012a7ca700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0124102090 0x7f012419e790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f011c007ee0 tx=0x7f011c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f0116ffd700 1 -- 172.21.15.42:0/2860592319 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c017070 con 0x7f0124102090 2023-12-12T09:57:10.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f0116ffd700 1 -- 172.21.15.42:0/2860592319 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f011c00f040 con 0x7f0124102090 2023-12-12T09:57:10.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f0116ffd700 1 -- 172.21.15.42:0/2860592319 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011c005580 con 0x7f0124102090 2023-12-12T09:57:10.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012419f540 con 0x7f0124102090 2023-12-12T09:57:10.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.536+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012419f9f0 con 0x7f0124102090 2023-12-12T09:57:10.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.537+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01241078d0 con 0x7f0124102090 2023-12-12T09:57:10.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.538+0000 7f0116ffd700 1 -- 172.21.15.42:0/2860592319 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f011c00db20 con 0x7f0124102090 2023-12-12T09:57:10.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.539+0000 7f0116ffd700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01100a1590 0x7f01100a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:10.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.539+0000 7f0129fc9700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01100a1590 0x7f01100a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:10.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.539+0000 7f0116ffd700 1 -- 172.21.15.42:0/2860592319 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f011c0cc730 con 0x7f0124102090 2023-12-12T09:57:10.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.539+0000 7f0129fc9700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01100a1590 0x7f01100a3a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0120006010 tx=0x7f0120000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:10.545 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.543+0000 7f0116ffd700 1 -- 172.21.15.42:0/2860592319 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f011c095a40 con 0x7f0124102090 2023-12-12T09:57:10.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.820+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f012404f290 con 0x7f0124102090 2023-12-12T09:57:10.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.822+0000 7f0116ffd700 1 -- 172.21.15.42:0/2860592319 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f011c099060 con 0x7f0124102090 2023-12-12T09:57:10.824 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2023-12-12T09:57:10.825 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQB2LnhlmRECMRAAdHjz1T9eYo50YM9HNCMXNQ== 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.825+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01100a1590 msgr2=0x7f01100a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.825+0000 7f012c22d700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01100a1590 0x7f01100a3a50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0120006010 tx=0x7f0120000bc0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.825+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0124102090 msgr2=0x7f012419e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.825+0000 7f012c22d700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0124102090 0x7f012419e790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f011c007ee0 tx=0x7f011c00d3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.825+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 shutdown_connections 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.825+0000 7f012c22d700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01100a1590 0x7f01100a3a50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.825+0000 7f012c22d700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01241004f0 0x7f012406a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.825+0000 7f012c22d700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0124102090 0x7f012419e790 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.825+0000 7f012c22d700 1 --2- 172.21.15.42:0/2860592319 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01241016f0 0x7f012419c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.826+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 >> 172.21.15.42:0/2860592319 conn(0x7f01240fba90 msgr2=0x7f0124104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.826+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 shutdown_connections 2023-12-12T09:57:10.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:10.826+0000 7f012c22d700 1 -- 172.21.15.42:0/2860592319 wait complete. 2023-12-12T09:57:11.248 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:57:11.249 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-12-12T09:57:11.249 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-12T09:57:11.291 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-12T09:57:11.291 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-12-12T09:57:11.291 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph mgr dump --format=json 2023-12-12T09:57:11.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:11 smithi173 ceph-mon[103308]: pgmap v278: 129 pgs: 129 active+clean; 625 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2023-12-12T09:57:11.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:11 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2860592319' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-12T09:57:11.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:11 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2860592319' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-12T09:57:11.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:11 smithi042 ceph-mon[98894]: pgmap v278: 129 pgs: 129 active+clean; 625 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2023-12-12T09:57:11.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:11 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2860592319' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-12T09:57:11.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:11 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2860592319' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-12T09:57:11.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:11 smithi111 ceph-mon[102565]: pgmap v278: 129 pgs: 129 active+clean; 625 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2023-12-12T09:57:11.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:11 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2860592319' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-12T09:57:11.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:11 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2860592319' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-12T09:57:11.931 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.267+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/80047148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c101680 msgr2=0x7f3b9c101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.267+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/80047148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c101680 0x7f3b9c101b00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3b88009990 tx=0x7f3b88009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.267+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/80047148 shutdown_connections 2023-12-12T09:57:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.267+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/80047148 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b9c1076f0 0x7f3b9c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.267+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/80047148 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9c102040 0x7f3b9c10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.271 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.267+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/80047148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c101680 0x7f3b9c101b00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.267+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/80047148 >> 172.21.15.42:0/80047148 conn(0x7f3b9c0787c0 msgr2=0x7f3b9c078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:13.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.267+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/80047148 shutdown_connections 2023-12-12T09:57:13.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.267+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/80047148 wait complete. 2023-12-12T09:57:13.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.268+0000 7f3ba2d0c700 1 Processor -- start 2023-12-12T09:57:13.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.268+0000 7f3ba2d0c700 1 -- start start 2023-12-12T09:57:13.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.268+0000 7f3ba2d0c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9c101680 0x7f3b9c19d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:13.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.268+0000 7f3ba2d0c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b9c102040 0x7f3b9c19dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:13.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba2d0c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c1076f0 0x7f3b9c197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba2d0c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c197ed0 con 0x7f3b9c1076f0 2023-12-12T09:57:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba2d0c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c198040 con 0x7f3b9c102040 2023-12-12T09:57:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba2d0c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c198200 con 0x7f3b9c101680 2023-12-12T09:57:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba12a9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c1076f0 0x7f3b9c197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba12a9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c1076f0 0x7f3b9c197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34050/0 (socket says 172.21.15.42:34050) 2023-12-12T09:57:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba12a9700 1 -- 172.21.15.42:0/3267829253 learned_addr learned my addr 172.21.15.42:0/3267829253 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:13.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba0aa8700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9c101680 0x7f3b9c19d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3b9bfff700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b9c102040 0x7f3b9c19dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba12a9700 1 -- 172.21.15.42:0/3267829253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9c101680 msgr2=0x7f3b9c19d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba12a9700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9c101680 0x7f3b9c19d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba12a9700 1 -- 172.21.15.42:0/3267829253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b9c102040 msgr2=0x7f3b9c19dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba12a9700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b9c102040 0x7f3b9c19dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba12a9700 1 -- 172.21.15.42:0/3267829253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b880095f0 con 0x7f3b9c1076f0 2023-12-12T09:57:13.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.269+0000 7f3ba12a9700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c1076f0 0x7f3b9c197860 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3b94007f10 tx=0x7f3b9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.270+0000 7f3b99ffb700 1 -- 172.21.15.42:0/3267829253 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94017070 con 0x7f3b9c1076f0 2023-12-12T09:57:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.270+0000 7f3b99ffb700 1 -- 172.21.15.42:0/3267829253 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b9400f040 con 0x7f3b9c1076f0 2023-12-12T09:57:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.270+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9c1984e0 con 0x7f3b9c1076f0 2023-12-12T09:57:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.270+0000 7f3b99ffb700 1 -- 172.21.15.42:0/3267829253 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940054b0 con 0x7f3b9c1076f0 2023-12-12T09:57:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.270+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c1a46c0 con 0x7f3b9c1076f0 2023-12-12T09:57:13.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.271+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9c109b80 con 0x7f3b9c1076f0 2023-12-12T09:57:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.272+0000 7f3b99ffb700 1 -- 172.21.15.42:0/3267829253 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f3b94004ad0 con 0x7f3b9c1076f0 2023-12-12T09:57:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.272+0000 7f3b99ffb700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3b8c0a1510 0x7f3b8c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.273+0000 7f3ba0aa8700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3b8c0a1510 0x7f3b8c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.273+0000 7f3b99ffb700 1 -- 172.21.15.42:0/3267829253 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3b940cc740 con 0x7f3b9c1076f0 2023-12-12T09:57:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.273+0000 7f3ba0aa8700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3b8c0a1510 0x7f3b8c0a39d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b90005950 tx=0x7f3b9000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.277+0000 7f3b99ffb700 1 -- 172.21.15.42:0/3267829253 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3b94095a50 con 0x7f3b9c1076f0 2023-12-12T09:57:13.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:13 smithi042 ceph-mon[98894]: pgmap v279: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.9 KiB/s wr, 18 op/s 2023-12-12T09:57:13.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.536+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3b9c102d10 con 0x7f3b9c1076f0 2023-12-12T09:57:13.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.541+0000 7f3b99ffb700 1 -- 172.21.15.42:0/3267829253 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260410 (secure 0 0 0) 0x7f3b94099070 con 0x7f3b9c1076f0 2023-12-12T09:57:13.546 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3b8c0a1510 msgr2=0x7f3b8c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3b8c0a1510 0x7f3b8c0a39d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3b90005950 tx=0x7f3b9000b410 comp rx=0 tx=0).stop 2023-12-12T09:57:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c1076f0 msgr2=0x7f3b9c197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:13.548 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c1076f0 0x7f3b9c197860 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3b94007f10 tx=0x7f3b9400d3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 shutdown_connections 2023-12-12T09:57:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3b8c0a1510 0x7f3b8c0a39d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3b9c1076f0 0x7f3b9c197860 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9c101680 0x7f3b9c19d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 --2- 172.21.15.42:0/3267829253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b9c102040 0x7f3b9c19dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 >> 172.21.15.42:0/3267829253 conn(0x7f3b9c0787c0 msgr2=0x7f3b9c1003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 shutdown_connections 2023-12-12T09:57:13.549 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:13.544+0000 7f3ba2d0c700 1 -- 172.21.15.42:0/3267829253 wait complete. 2023-12-12T09:57:13.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:13 smithi173 ceph-mon[103308]: pgmap v279: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.9 KiB/s wr, 18 op/s 2023-12-12T09:57:13.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:13 smithi111 ceph-mon[102565]: pgmap v279: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.9 KiB/s wr, 18 op/s 2023-12-12T09:57:13.995 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":2637927159},{"type":"v1","addr":"172.21.15.42:6801","nonce":2637927159}]},"active_addr":"172.21.15.42:6801/2637927159","active_change":"2023-12-12T09:50:27.727371+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.42:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3987075096}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":2112736882}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3046151373}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3552425799}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":320676444}]}]} 2023-12-12T09:57:14.000 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-12-12T09:57:14.000 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-12-12T09:57:14.000 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd dump --format=json 2023-12-12T09:57:14.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:14 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3267829253' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-12T09:57:14.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:14 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3267829253' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-12T09:57:14.604 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:14.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:14 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3267829253' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-12T09:57:15.388 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:15 smithi042 ceph-mon[98894]: pgmap v280: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2023-12-12T09:57:15.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:15 smithi173 ceph-mon[103308]: pgmap v280: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2023-12-12T09:57:15.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:15 smithi111 ceph-mon[102565]: pgmap v280: 129 pgs: 129 active+clean; 626 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 7.9 KiB/s wr, 20 op/s 2023-12-12T09:57:15.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/419531208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101a40 msgr2=0x7f5d3810d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:15.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/419531208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101a40 0x7f5d3810d3a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000b0a0 tx=0x7f5d3000b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:15.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/419531208 shutdown_connections 2023-12-12T09:57:15.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/419531208 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101a40 0x7f5d3810d3a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:15.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/419531208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d38105500 0x7f5d381058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:15.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/419531208 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38101080 0x7f5d38101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:15.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/419531208 >> 172.21.15.42:0/419531208 conn(0x7f5d380757f0 msgr2=0x7f5d38075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:15.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/419531208 shutdown_connections 2023-12-12T09:57:15.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/419531208 wait complete. 2023-12-12T09:57:15.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.852+0000 7f5d3dc1e700 1 Processor -- start 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d3dc1e700 1 -- start start 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d3dc1e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101080 0x7f5d381989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d3dc1e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d38101a40 0x7f5d38198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d3dc1e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38105500 0x7f5d381a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d3dc1e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d38199750 con 0x7f5d38101080 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d3dc1e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d381a3f60 con 0x7f5d38105500 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d3dc1e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d381a40a0 con 0x7f5d38101a40 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d377fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101080 0x7f5d381989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d377fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101080 0x7f5d381989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34060/0 (socket says 172.21.15.42:34060) 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d377fe700 1 -- 172.21.15.42:0/2213011880 learned_addr learned my addr 172.21.15.42:0/2213011880 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d37fff700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38105500 0x7f5d381a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:15.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d36ffd700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d38101a40 0x7f5d38198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d377fe700 1 -- 172.21.15.42:0/2213011880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d38101a40 msgr2=0x7f5d38198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d377fe700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d38101a40 0x7f5d38198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d377fe700 1 -- 172.21.15.42:0/2213011880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38105500 msgr2=0x7f5d381a3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d377fe700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38105500 0x7f5d381a3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d377fe700 1 -- 172.21.15.42:0/2213011880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d30009d00 con 0x7f5d38101080 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d37fff700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38105500 0x7f5d381a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.853+0000 7f5d36ffd700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d38101a40 0x7f5d38198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.854+0000 7f5d377fe700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101080 0x7f5d381989a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800c920 tx=0x7f5d2800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.854+0000 7f5d34ff9700 1 -- 172.21.15.42:0/2213011880 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28007b00 con 0x7f5d38101080 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.854+0000 7f5d34ff9700 1 -- 172.21.15.42:0/2213011880 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d28004cf0 con 0x7f5d38101080 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.854+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d381a4330 con 0x7f5d38101080 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.854+0000 7f5d34ff9700 1 -- 172.21.15.42:0/2213011880 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28005610 con 0x7f5d38101080 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.854+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d381a4830 con 0x7f5d38101080 2023-12-12T09:57:15.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.855+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d38108af0 con 0x7f5d38101080 2023-12-12T09:57:15.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.856+0000 7f5d34ff9700 1 -- 172.21.15.42:0/2213011880 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f5d28004750 con 0x7f5d38101080 2023-12-12T09:57:15.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.857+0000 7f5d34ff9700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d200a1510 0x7f5d200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:15.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.857+0000 7f5d36ffd700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d200a1510 0x7f5d200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:15.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.857+0000 7f5d34ff9700 1 -- 172.21.15.42:0/2213011880 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5d280cd070 con 0x7f5d38101080 2023-12-12T09:57:15.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.857+0000 7f5d36ffd700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d200a1510 0x7f5d200a39d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005fd0 tx=0x7f5d2c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:15.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:15.860+0000 7f5d34ff9700 1 -- 172.21.15.42:0/2213011880 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5d28096320 con 0x7f5d38101080 2023-12-12T09:57:16.062 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.060+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d38068af0 con 0x7f5d38101080 2023-12-12T09:57:16.063 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.061+0000 7f5d34ff9700 1 -- 172.21.15.42:0/2213011880 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7f5d2801d020 con 0x7f5d38101080 2023-12-12T09:57:16.064 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:16.064 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d200a1510 msgr2=0x7f5d200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d200a1510 0x7f5d200a39d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c005fd0 tx=0x7f5d2c005ee0 comp rx=0 tx=0).stop 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101080 msgr2=0x7f5d381989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101080 0x7f5d381989a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800c920 tx=0x7f5d2800cce0 comp rx=0 tx=0).stop 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 shutdown_connections 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5d200a1510 0x7f5d200a39d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5d38105500 0x7f5d381a3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d38101080 0x7f5d381989a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 --2- 172.21.15.42:0/2213011880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d38101a40 0x7f5d38198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:16.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.063+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 >> 172.21.15.42:0/2213011880 conn(0x7f5d380757f0 msgr2=0x7f5d380fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:16.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.064+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 shutdown_connections 2023-12-12T09:57:16.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:16.064+0000 7f5d3dc1e700 1 -- 172.21.15.42:0/2213011880 wait complete. 2023-12-12T09:57:16.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:16 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2213011880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:16.476 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-12-12T09:57:16.476 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd dump --format=json 2023-12-12T09:57:16.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:16 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2213011880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:16.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:16 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2213011880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:17.080 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:17.596 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:17 smithi042 ceph-mon[98894]: pgmap v281: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.0 KiB/s wr, 23 op/s 2023-12-12T09:57:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:17 smithi111 ceph-mon[102565]: pgmap v281: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.0 KiB/s wr, 23 op/s 2023-12-12T09:57:17.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:17 smithi173 ceph-mon[103308]: pgmap v281: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 9.0 KiB/s wr, 23 op/s 2023-12-12T09:57:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.305+0000 7f7f68e51700 1 -- 172.21.15.42:0/2355003089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 msgr2=0x7f7f641009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.305+0000 7f7f68e51700 1 --2- 172.21.15.42:0/2355003089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 0x7f7f641009b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0099c0 tx=0x7f7f4c009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.305+0000 7f7f68e51700 1 -- 172.21.15.42:0/2355003089 shutdown_connections 2023-12-12T09:57:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.305+0000 7f7f68e51700 1 --2- 172.21.15.42:0/2355003089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f64100ef0 0x7f7f6410d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.305+0000 7f7f68e51700 1 --2- 172.21.15.42:0/2355003089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f64106560 0x7f7f64106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.305+0000 7f7f68e51700 1 --2- 172.21.15.42:0/2355003089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 0x7f7f641009b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.305+0000 7f7f68e51700 1 -- 172.21.15.42:0/2355003089 >> 172.21.15.42:0/2355003089 conn(0x7f7f64076570 msgr2=0x7f7f64076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.305+0000 7f7f68e51700 1 -- 172.21.15.42:0/2355003089 shutdown_connections 2023-12-12T09:57:18.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.305+0000 7f7f68e51700 1 -- 172.21.15.42:0/2355003089 wait complete. 2023-12-12T09:57:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.306+0000 7f7f68e51700 1 Processor -- start 2023-12-12T09:57:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.306+0000 7f7f68e51700 1 -- start start 2023-12-12T09:57:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.306+0000 7f7f68e51700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 0x7f7f641965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:18.310 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.306+0000 7f7f68e51700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f64100ef0 0x7f7f64196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.306+0000 7f7f68e51700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f64106560 0x7f7f64197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.306+0000 7f7f68e51700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f641a5d30 con 0x7f7f64100530 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.306+0000 7f7f68e51700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f641a5ea0 con 0x7f7f64100ef0 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.306+0000 7f7f68e51700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f641a6060 con 0x7f7f64106560 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f6259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 0x7f7f641965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f6259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 0x7f7f641965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34072/0 (socket says 172.21.15.42:34072) 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f6259c700 1 -- 172.21.15.42:0/3269511583 learned_addr learned my addr 172.21.15.42:0/3269511583 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f62d9d700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f64106560 0x7f7f64197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f5bfff700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f64100ef0 0x7f7f64196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f6259c700 1 -- 172.21.15.42:0/3269511583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f64106560 msgr2=0x7f7f64197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f6259c700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f64106560 0x7f7f64197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f6259c700 1 -- 172.21.15.42:0/3269511583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f64100ef0 msgr2=0x7f7f64196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f6259c700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f64100ef0 0x7f7f64196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f6259c700 1 -- 172.21.15.42:0/3269511583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c0095f0 con 0x7f7f64100530 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f6259c700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 0x7f7f641965e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400d950 tx=0x7f7f5400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:18.311 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f5b7fe700 1 -- 172.21.15.42:0/3269511583 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400f890 con 0x7f7f64100530 2023-12-12T09:57:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f5b7fe700 1 -- 172.21.15.42:0/3269511583 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f54010460 con 0x7f7f64100530 2023-12-12T09:57:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f5b7fe700 1 -- 172.21.15.42:0/3269511583 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5400e610 con 0x7f7f64100530 2023-12-12T09:57:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f641a6340 con 0x7f7f64100530 2023-12-12T09:57:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.307+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f641a67c0 con 0x7f7f64100530 2023-12-12T09:57:18.312 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.308+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f64108a10 con 0x7f7f64100530 2023-12-12T09:57:18.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.310+0000 7f7f5b7fe700 1 -- 172.21.15.42:0/3269511583 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7f540108d0 con 0x7f7f64100530 2023-12-12T09:57:18.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.310+0000 7f7f5b7fe700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7f500a1510 0x7f7f500a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:18.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.310+0000 7f7f5bfff700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7f500a1510 0x7f7f500a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:18.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.310+0000 7f7f5b7fe700 1 -- 172.21.15.42:0/3269511583 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7f540ccdf0 con 0x7f7f64100530 2023-12-12T09:57:18.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.311+0000 7f7f5bfff700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7f500a1510 0x7f7f500a39d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009990 tx=0x7f7f4c003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:18.316 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.314+0000 7f7f5b7fe700 1 -- 172.21.15.42:0/3269511583 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7f54096100 con 0x7f7f64100530 2023-12-12T09:57:18.518 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.516+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f64101bd0 con 0x7f7f64100530 2023-12-12T09:57:18.519 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.517+0000 7f7f5b7fe700 1 -- 172.21.15.42:0/3269511583 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+23949 (secure 0 0 0) 0x7f7f54016090 con 0x7f7f64100530 2023-12-12T09:57:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:18.520 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":87,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:56:42.948539+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.519+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7f500a1510 msgr2=0x7f7f500a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.519+0000 7f7f68e51700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7f500a1510 0x7f7f500a39d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009990 tx=0x7f7f4c003680 comp rx=0 tx=0).stop 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.519+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 msgr2=0x7f7f641965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.519+0000 7f7f68e51700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 0x7f7f641965e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7f5400d950 tx=0x7f7f5400dc60 comp rx=0 tx=0).stop 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.520+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 shutdown_connections 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.520+0000 7f7f68e51700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7f500a1510 0x7f7f500a39d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.520+0000 7f7f68e51700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f64106560 0x7f7f64197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.520+0000 7f7f68e51700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f64100530 0x7f7f641965e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.520+0000 7f7f68e51700 1 --2- 172.21.15.42:0/3269511583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7f64100ef0 0x7f7f64196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.520+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 >> 172.21.15.42:0/3269511583 conn(0x7f7f64076570 msgr2=0x7f7f640ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:18.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.520+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 shutdown_connections 2023-12-12T09:57:18.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:18.520+0000 7f7f68e51700 1 -- 172.21.15.42:0/3269511583 wait complete. 2023-12-12T09:57:19.054 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-12-12T09:57:19.055 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-12-12T09:57:19.055 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-12-12T09:57:19.055 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-12-12T09:57:19.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-12-12T09:57:19.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-12-12T09:57:19.056 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-12-12T09:57:19.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-12-12T09:57:19.057 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-12-12T09:57:19.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-12-12T09:57:19.058 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-12-12T09:57:19.059 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-12-12T09:57:19.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:19 smithi042 ceph-mon[98894]: pgmap v282: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.8 KiB/s wr, 22 op/s 2023-12-12T09:57:19.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:19 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3269511583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:19 smithi111 ceph-mon[102565]: pgmap v282: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.8 KiB/s wr, 22 op/s 2023-12-12T09:57:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:19 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3269511583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:19 smithi173 ceph-mon[103308]: pgmap v282: 129 pgs: 129 active+clean; 634 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.8 KiB/s wr, 22 op/s 2023-12-12T09:57:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:19 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3269511583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:20.270 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:20.423 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:20.423 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:20.731 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:20.845 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:20.969 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:21.388 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:21.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:21 smithi042 ceph-mon[98894]: pgmap v283: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2023-12-12T09:57:21.433 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:21 smithi111 ceph-mon[102565]: pgmap v283: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2023-12-12T09:57:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:21 smithi173 ceph-mon[103308]: pgmap v283: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.1 KiB/s wr, 16 op/s 2023-12-12T09:57:23.312 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:23.367 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:23 smithi042 ceph-mon[98894]: pgmap v284: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-12-12T09:57:23.733 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:23.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:23 smithi111 ceph-mon[102565]: pgmap v284: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-12-12T09:57:23.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:23 smithi173 ceph-mon[103308]: pgmap v284: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2023-12-12T09:57:25.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.203+0000 7f340de02700 1 -- 172.21.15.42:0/1060254500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34081004c0 msgr2=0x7f3408100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.203+0000 7f340de02700 1 --2- 172.21.15.42:0/1060254500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34081004c0 0x7f3408100940 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0099c0 tx=0x7f33fc009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.203+0000 7f340de02700 1 -- 172.21.15.42:0/1060254500 shutdown_connections 2023-12-12T09:57:25.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.203+0000 7f340de02700 1 --2- 172.21.15.42:0/1060254500 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3408105470 0x7f3408105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.203+0000 7f340de02700 1 --2- 172.21.15.42:0/1060254500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3408100e80 0x7f340810d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.203+0000 7f340de02700 1 --2- 172.21.15.42:0/1060254500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34081004c0 0x7f3408100940 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.203+0000 7f340de02700 1 -- 172.21.15.42:0/1060254500 >> 172.21.15.42:0/1060254500 conn(0x7f34080774a0 msgr2=0x7f34080778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:25.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.203+0000 7f340de02700 1 -- 172.21.15.42:0/1060254500 shutdown_connections 2023-12-12T09:57:25.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.203+0000 7f340de02700 1 -- 172.21.15.42:0/1060254500 wait complete. 2023-12-12T09:57:25.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.204+0000 7f340de02700 1 Processor -- start 2023-12-12T09:57:25.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.204+0000 7f340de02700 1 -- start start 2023-12-12T09:57:25.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.204+0000 7f340de02700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34081004c0 0x7f3408197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.204+0000 7f340de02700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3408100e80 0x7f3408197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.204+0000 7f340de02700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3408105470 0x7f340819c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f340de02700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340819c900 con 0x7f34081004c0 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f340de02700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340819ca70 con 0x7f3408105470 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f340de02700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340819cc30 con 0x7f3408100e80 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f34077fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34081004c0 0x7f3408197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f34077fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34081004c0 0x7f3408197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34100/0 (socket says 172.21.15.42:34100) 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f34077fe700 1 -- 172.21.15.42:0/3275300486 learned_addr learned my addr 172.21.15.42:0/3275300486 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f3406ffd700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3408100e80 0x7f3408197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f34077fe700 1 -- 172.21.15.42:0/3275300486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3408100e80 msgr2=0x7f3408197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f3407fff700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3408105470 0x7f340819c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f34077fe700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3408100e80 0x7f3408197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f34077fe700 1 -- 172.21.15.42:0/3275300486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3408105470 msgr2=0x7f340819c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f34077fe700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3408105470 0x7f340819c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f34077fe700 1 -- 172.21.15.42:0/3275300486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc0095f0 con 0x7f34081004c0 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f3407fff700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3408105470 0x7f340819c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f34077fe700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34081004c0 0x7f3408197a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d980 tx=0x7f33f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f3404ff9700 1 -- 172.21.15.42:0/3275300486 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8004020 con 0x7f34081004c0 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340819cf10 con 0x7f34081004c0 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f3404ff9700 1 -- 172.21.15.42:0/3275300486 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f33f8009730 con 0x7f34081004c0 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f3404ff9700 1 -- 172.21.15.42:0/3275300486 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f8003a30 con 0x7f34081004c0 2023-12-12T09:57:25.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.205+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34081a47b0 con 0x7f34081004c0 2023-12-12T09:57:25.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.206+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3408110710 con 0x7f34081004c0 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.208+0000 7f3404ff9700 1 -- 172.21.15.42:0/3275300486 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f33f80098a0 con 0x7f34081004c0 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.208+0000 7f3404ff9700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f33f00a1510 0x7f33f00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.208+0000 7f3404ff9700 1 -- 172.21.15.42:0/3275300486 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f33f80cd490 con 0x7f34081004c0 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.208+0000 7f3406ffd700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f33f00a1510 0x7f33f00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.209+0000 7f3404ff9700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] conn(0x7f33f00ac090 0x7f33f00ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.209+0000 7f3404ff9700 1 -- 172.21.15.42:0/3275300486 --> [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33f00aeba0 con 0x7f33f00ac090 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.209+0000 7f3404ff9700 1 -- 172.21.15.42:0/3275300486 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33f80cd8a0 con 0x7f34081004c0 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.209+0000 7f3407fff700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] conn(0x7f33f00ac090 0x7f33f00ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.209+0000 7f3406ffd700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f33f00a1510 0x7f33f00a39d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00b580 tx=0x7f33fc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.209+0000 7f3407fff700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] conn(0x7f33f00ac090 0x7f33f00ae4d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.210+0000 7f3404ff9700 1 -- 172.21.15.42:0/3275300486 <== osd.2 v2:172.21.15.173:6800/3785124415 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f33f00aeba0 con 0x7f33f00ac090 2023-12-12T09:57:25.230 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.229+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 --> [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3408101bb0 con 0x7f33f00ac090 2023-12-12T09:57:25.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.229+0000 7f3404ff9700 1 -- 172.21.15.42:0/3275300486 <== osd.2 v2:172.21.15.173:6800/3785124415 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3408101bb0 con 0x7f33f00ac090 2023-12-12T09:57:25.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.229+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 >> [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] conn(0x7f33f00ac090 msgr2=0x7f33f00ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.229+0000 7f340de02700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] conn(0x7f33f00ac090 0x7f33f00ae4d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.230+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f33f00a1510 msgr2=0x7f33f00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.230+0000 7f340de02700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f33f00a1510 0x7f33f00a39d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00b580 tx=0x7f33fc005a90 comp rx=0 tx=0).stop 2023-12-12T09:57:25.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.230+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34081004c0 msgr2=0x7f3408197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.230+0000 7f340de02700 1 --2- 172.21.15.42:0/3275300486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34081004c0 0x7f3408197a10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f33f800d980 tx=0x7f33f800dc90 comp rx=0 tx=0).stop 2023-12-12T09:57:25.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.230+0000 7f3407fff700 1 -- 172.21.15.42:0/3275300486 reap_dead start 2023-12-12T09:57:25.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.230+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 shutdown_connections 2023-12-12T09:57:25.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.230+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 >> 172.21.15.42:0/3275300486 conn(0x7f34080774a0 msgr2=0x7f3408069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:25.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.230+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 shutdown_connections 2023-12-12T09:57:25.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.230+0000 7f340de02700 1 -- 172.21.15.42:0/3275300486 wait complete. 2023-12-12T09:57:25.495 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:25 smithi042 ceph-mon[98894]: pgmap v285: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-12-12T09:57:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 -- 172.21.15.42:0/3167249502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb80069800 msgr2=0x7fdb80069be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 --2- 172.21.15.42:0/3167249502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb80069800 0x7fdb80069be0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009990 tx=0x7fdb70009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 -- 172.21.15.42:0/3167249502 shutdown_connections 2023-12-12T09:57:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 --2- 172.21.15.42:0/3167249502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb8010e070 0x7fdb80110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 --2- 172.21.15.42:0/3167249502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb80069800 0x7fdb80069be0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 --2- 172.21.15.42:0/3167249502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb8006a120 0x7fdb8010db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 -- 172.21.15.42:0/3167249502 >> 172.21.15.42:0/3167249502 conn(0x7fdb800774a0 msgr2=0x7fdb800778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 -- 172.21.15.42:0/3167249502 shutdown_connections 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 -- 172.21.15.42:0/3167249502 wait complete. 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.531+0000 7fdb85bca700 1 Processor -- start 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb85bca700 1 -- start start 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb85bca700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80069800 0x7fdb8019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb85bca700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb8006a120 0x7fdb8019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb85bca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8010e070 0x7fdb80199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb85bca700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8019a0d0 con 0x7fdb8010e070 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb85bca700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8019a240 con 0x7fdb8006a120 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb85bca700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8019a400 con 0x7fdb80069800 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8010e070 0x7fdb80199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8010e070 0x7fdb80199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34120/0 (socket says 172.21.15.42:34120) 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7ffff700 1 -- 172.21.15.42:0/1106317274 learned_addr learned my addr 172.21.15.42:0/1106317274 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7effd700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb8006a120 0x7fdb8019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7f7fe700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80069800 0x7fdb8019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7ffff700 1 -- 172.21.15.42:0/1106317274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80069800 msgr2=0x7fdb8019f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7ffff700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb80069800 0x7fdb8019f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7ffff700 1 -- 172.21.15.42:0/1106317274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb8006a120 msgr2=0x7fdb8019fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7ffff700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb8006a120 0x7fdb8019fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.532+0000 7fdb7ffff700 1 -- 172.21.15.42:0/1106317274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb700095f0 con 0x7fdb8010e070 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.533+0000 7fdb7ffff700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8010e070 0x7fdb80199a70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800ada0 tx=0x7fdb7800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.533+0000 7fdb7cff9700 1 -- 172.21.15.42:0/1106317274 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800f070 con 0x7fdb8010e070 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.533+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8019a6e0 con 0x7fdb8010e070 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.533+0000 7fdb7cff9700 1 -- 172.21.15.42:0/1106317274 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb78007b00 con 0x7fdb8010e070 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.533+0000 7fdb7cff9700 1 -- 172.21.15.42:0/1106317274 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb780053a0 con 0x7fdb8010e070 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.533+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb801a6920 con 0x7fdb8010e070 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.533+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb801139b0 con 0x7fdb8010e070 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.536+0000 7fdb7cff9700 1 -- 172.21.15.42:0/1106317274 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fdb7800c980 con 0x7fdb8010e070 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.537+0000 7fdb7cff9700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdb680a1510 0x7fdb680a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.537+0000 7fdb7cff9700 1 -- 172.21.15.42:0/1106317274 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdb780ccfc0 con 0x7fdb8010e070 2023-12-12T09:57:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.537+0000 7fdb7f7fe700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdb680a1510 0x7fdb680a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.537+0000 7fdb7cff9700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] conn(0x7fdb680ac090 0x7fdb680ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.537+0000 7fdb7cff9700 1 -- 172.21.15.42:0/1106317274 --> [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb680aeba0 con 0x7fdb680ac090 2023-12-12T09:57:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.537+0000 7fdb7cff9700 1 -- 172.21.15.42:0/1106317274 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb780cd330 con 0x7fdb8010e070 2023-12-12T09:57:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.537+0000 7fdb7effd700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] conn(0x7fdb680ac090 0x7fdb680ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.537+0000 7fdb7f7fe700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdb680a1510 0x7fdb680a39d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb70006010 tx=0x7fdb7001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.538+0000 7fdb7effd700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] conn(0x7fdb680ac090 0x7fdb680ae4d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.554 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.538+0000 7fdb7cff9700 1 -- 172.21.15.42:0/1106317274 <== osd.9 v2:172.21.15.42:6826/1755856143 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fdb680aeba0 con 0x7fdb680ac090 2023-12-12T09:57:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.556+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 --> [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdb80061cd0 con 0x7fdb680ac090 2023-12-12T09:57:25.558 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.556+0000 7fdb7cff9700 1 -- 172.21.15.42:0/1106317274 <== osd.9 v2:172.21.15.42:6826/1755856143 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdb80061cd0 con 0x7fdb680ac090 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] conn(0x7fdb680ac090 msgr2=0x7fdb680ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] conn(0x7fdb680ac090 0x7fdb680ae4d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdb680a1510 msgr2=0x7fdb680a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdb680a1510 0x7fdb680a39d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb70006010 tx=0x7fdb7001a040 comp rx=0 tx=0).stop 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8010e070 msgr2=0x7fdb80199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 --2- 172.21.15.42:0/1106317274 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdb8010e070 0x7fdb80199a70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800ada0 tx=0x7fdb7800c330 comp rx=0 tx=0).stop 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb7ffff700 1 -- 172.21.15.42:0/1106317274 reap_dead start 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 shutdown_connections 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 >> 172.21.15.42:0/1106317274 conn(0x7fdb800774a0 msgr2=0x7fdb8010f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:25.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 shutdown_connections 2023-12-12T09:57:25.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.557+0000 7fdb85bca700 1 -- 172.21.15.42:0/1106317274 wait complete. 2023-12-12T09:57:25.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.413+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2195695193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4ac101680 msgr2=0x7ff4ac101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.413+0000 7ff4b0b0c700 1 --2- 172.21.15.42:0/2195695193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4ac101680 0x7ff4ac101b00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff494009990 tx=0x7ff494009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.413+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2195695193 shutdown_connections 2023-12-12T09:57:25.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.413+0000 7ff4b0b0c700 1 --2- 172.21.15.42:0/2195695193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ac1065a0 0x7ff4ac106980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.413+0000 7ff4b0b0c700 1 --2- 172.21.15.42:0/2195695193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ac102040 0x7ff4ac10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.413+0000 7ff4b0b0c700 1 --2- 172.21.15.42:0/2195695193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4ac101680 0x7ff4ac101b00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.413+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2195695193 >> 172.21.15.42:0/2195695193 conn(0x7ff4ac078730 msgr2=0x7ff4ac078b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.413+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2195695193 shutdown_connections 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.413+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2195695193 wait complete. 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4b0b0c700 1 Processor -- start 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4b0b0c700 1 -- start start 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4b0b0c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4ac101680 0x7ff4ac19d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4b0b0c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ac102040 0x7ff4ac19dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4b0b0c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ac1065a0 0x7ff4ac197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4b0b0c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac197e90 con 0x7ff4ac101680 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4b0b0c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac198000 con 0x7ff4ac102040 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4b0b0c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac1981c0 con 0x7ff4ac1065a0 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4aa59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4ac101680 0x7ff4ac19d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4aa59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4ac101680 0x7ff4ac19d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34084/0 (socket says 172.21.15.42:34084) 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4aa59c700 1 -- 172.21.15.42:0/2561513015 learned_addr learned my addr 172.21.15.42:0/2561513015 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.414+0000 7ff4aad9d700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ac1065a0 0x7ff4ac197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4a9d9b700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ac102040 0x7ff4ac19dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4aa59c700 1 -- 172.21.15.42:0/2561513015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ac1065a0 msgr2=0x7ff4ac197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4aa59c700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ac1065a0 0x7ff4ac197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4aa59c700 1 -- 172.21.15.42:0/2561513015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ac102040 msgr2=0x7ff4ac19dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4aa59c700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ac102040 0x7ff4ac19dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4aa59c700 1 -- 172.21.15.42:0/2561513015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4940095f0 con 0x7ff4ac101680 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4aad9d700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ac1065a0 0x7ff4ac197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4a9d9b700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff4ac102040 0x7ff4ac19dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4aa59c700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4ac101680 0x7ff4ac19d690 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c970 tx=0x7ff49c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4a37fe700 1 -- 172.21.15.42:0/2561513015 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c004220 con 0x7ff4ac101680 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4a37fe700 1 -- 172.21.15.42:0/2561513015 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49c007d60 con 0x7ff4ac101680 2023-12-12T09:57:25.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ac1984a0 con 0x7ff4ac101680 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4a37fe700 1 -- 172.21.15.42:0/2561513015 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c003d00 con 0x7ff4ac101680 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.415+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ac1a4680 con 0x7ff4ac101680 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.416+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff4ac111780 con 0x7ff4ac101680 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.417+0000 7ff4a37fe700 1 -- 172.21.15.42:0/2561513015 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff49c004380 con 0x7ff4ac101680 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.418+0000 7ff4a37fe700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4980a1510 0x7ff4980a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.418+0000 7ff4a37fe700 1 -- 172.21.15.42:0/2561513015 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff49c0cd020 con 0x7ff4ac101680 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.418+0000 7ff4a9d9b700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4980a1510 0x7ff4980a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.418+0000 7ff4a37fe700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] conn(0x7ff4980ac090 0x7ff4980ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.418+0000 7ff4a37fe700 1 -- 172.21.15.42:0/2561513015 --> [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4980aeba0 con 0x7ff4980ac090 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.418+0000 7ff4a37fe700 1 -- 172.21.15.42:0/2561513015 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49c0cd430 con 0x7ff4ac101680 2023-12-12T09:57:25.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.418+0000 7ff4a9d9b700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4980a1510 0x7ff4980a39d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff494006010 tx=0x7ff49401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.418+0000 7ff4aad9d700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] conn(0x7ff4980ac090 0x7ff4980ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.419+0000 7ff4aad9d700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] conn(0x7ff4980ac090 0x7ff4980ae4d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.420+0000 7ff4a37fe700 1 -- 172.21.15.42:0/2561513015 <== osd.8 v2:172.21.15.173:6816/1898117914 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff4980aeba0 con 0x7ff4980ac090 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.437+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 --> [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff4ac061cd0 con 0x7ff4980ac090 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.437+0000 7ff4a37fe700 1 -- 172.21.15.42:0/2561513015 <== osd.8 v2:172.21.15.173:6816/1898117914 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff4ac061cd0 con 0x7ff4980ac090 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 >> [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] conn(0x7ff4980ac090 msgr2=0x7ff4980ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] conn(0x7ff4980ac090 0x7ff4980ae4d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4980a1510 msgr2=0x7ff4980a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff4980a1510 0x7ff4980a39d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff494006010 tx=0x7ff49401a040 comp rx=0 tx=0).stop 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4ac101680 msgr2=0x7ff4ac19d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 --2- 172.21.15.42:0/2561513015 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4ac101680 0x7ff4ac19d690 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c970 tx=0x7ff49c00cd30 comp rx=0 tx=0).stop 2023-12-12T09:57:25.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4aad9d700 1 -- 172.21.15.42:0/2561513015 reap_dead start 2023-12-12T09:57:25.661 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 shutdown_connections 2023-12-12T09:57:25.661 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 >> 172.21.15.42:0/2561513015 conn(0x7ff4ac078730 msgr2=0x7ff4ac0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:25.661 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 shutdown_connections 2023-12-12T09:57:25.661 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:24.438+0000 7ff4b0b0c700 1 -- 172.21.15.42:0/2561513015 wait complete. 2023-12-12T09:57:25.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:25 smithi111 ceph-mon[102565]: pgmap v285: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-12-12T09:57:25.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:25 smithi173 ceph-mon[103308]: pgmap v285: 129 pgs: 129 active+clean; 638 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2023-12-12T09:57:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 -- 172.21.15.42:0/4019490489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa48100e80 msgr2=0x7faa4810d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 --2- 172.21.15.42:0/4019490489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa48100e80 0x7faa4810d370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faa4000b0d0 tx=0x7faa4000b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 -- 172.21.15.42:0/4019490489 shutdown_connections 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 --2- 172.21.15.42:0/4019490489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa48100e80 0x7faa4810d370 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 --2- 172.21.15.42:0/4019490489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48105470 0x7faa48105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 --2- 172.21.15.42:0/4019490489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa481004c0 0x7faa48100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 -- 172.21.15.42:0/4019490489 >> 172.21.15.42:0/4019490489 conn(0x7faa480774a0 msgr2=0x7faa480778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 -- 172.21.15.42:0/4019490489 shutdown_connections 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 -- 172.21.15.42:0/4019490489 wait complete. 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 Processor -- start 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.586+0000 7faa4ebb7700 1 -- start start 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4ebb7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa481004c0 0x7faa48102100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4ebb7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa48100e80 0x7faa48102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4ebb7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48105470 0x7faa48102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4ebb7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa481a5e50 con 0x7faa481004c0 2023-12-12T09:57:25.852 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4ebb7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa481a5fc0 con 0x7faa48100e80 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4ebb7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa481a6180 con 0x7faa48105470 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4c953700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa481004c0 0x7faa48102100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4c953700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa481004c0 0x7faa48102100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34132/0 (socket says 172.21.15.42:34132) 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4c953700 1 -- 172.21.15.42:0/1956425268 learned_addr learned my addr 172.21.15.42:0/1956425268 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4d154700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48105470 0x7faa48102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa47fff700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa48100e80 0x7faa48102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4c953700 1 -- 172.21.15.42:0/1956425268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48105470 msgr2=0x7faa48102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4c953700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48105470 0x7faa48102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4c953700 1 -- 172.21.15.42:0/1956425268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa48100e80 msgr2=0x7faa48102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4c953700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa48100e80 0x7faa48102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4c953700 1 -- 172.21.15.42:0/1956425268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa40009d00 con 0x7faa481004c0 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4d154700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa48105470 0x7faa48102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4c953700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa481004c0 0x7faa48102100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00d980 tx=0x7faa3c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.853 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa45ffb700 1 -- 172.21.15.42:0/1956425268 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c00de60 con 0x7faa481004c0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa45ffb700 1 -- 172.21.15.42:0/1956425268 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa3c0044e0 con 0x7faa481004c0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa481a6460 con 0x7faa481004c0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.587+0000 7faa45ffb700 1 -- 172.21.15.42:0/1956425268 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c003be0 con 0x7faa481004c0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.588+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa481a6960 con 0x7faa481004c0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.588+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faa481106c0 con 0x7faa481004c0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.590+0000 7faa45ffb700 1 -- 172.21.15.42:0/1956425268 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7faa3c009bb0 con 0x7faa481004c0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.590+0000 7faa45ffb700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faa380a1510 0x7faa380a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.590+0000 7faa47fff700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faa380a1510 0x7faa380a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.590+0000 7faa45ffb700 1 -- 172.21.15.42:0/1956425268 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7faa3c0cd540 con 0x7faa481004c0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.590+0000 7faa45ffb700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] conn(0x7faa380ac090 0x7faa380ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.590+0000 7faa45ffb700 1 -- 172.21.15.42:0/1956425268 --> [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faa380aeba0 con 0x7faa380ac090 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.590+0000 7faa45ffb700 1 -- 172.21.15.42:0/1956425268 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa3c0cd950 con 0x7faa481004c0 2023-12-12T09:57:25.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.591+0000 7faa47fff700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faa380a1510 0x7faa380a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faa34005950 tx=0x7faa340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.591+0000 7faa4d154700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] conn(0x7faa380ac090 0x7faa380ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.591+0000 7faa4d154700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] conn(0x7faa380ac090 0x7faa380ae4d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.592+0000 7faa45ffb700 1 -- 172.21.15.42:0/1956425268 <== osd.4 v2:172.21.15.111:6808/724947563 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7faa380aeba0 con 0x7faa380ac090 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.609+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 --> [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faa48101bb0 con 0x7faa380ac090 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa45ffb700 1 -- 172.21.15.42:0/1956425268 <== osd.4 v2:172.21.15.111:6808/724947563 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7faa48101bb0 con 0x7faa380ac090 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 >> [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] conn(0x7faa380ac090 msgr2=0x7faa380ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4ebb7700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] conn(0x7faa380ac090 0x7faa380ae4d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faa380a1510 msgr2=0x7faa380a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4ebb7700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faa380a1510 0x7faa380a39d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faa34005950 tx=0x7faa340058e0 comp rx=0 tx=0).stop 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa481004c0 msgr2=0x7faa48102100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4ebb7700 1 --2- 172.21.15.42:0/1956425268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faa481004c0 0x7faa48102100 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00d980 tx=0x7faa3c00dd40 comp rx=0 tx=0).stop 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4d154700 1 -- 172.21.15.42:0/1956425268 reap_dead start 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 shutdown_connections 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 >> 172.21.15.42:0/1956425268 conn(0x7faa480774a0 msgr2=0x7faa48069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:25.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.610+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 shutdown_connections 2023-12-12T09:57:25.856 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:25.611+0000 7faa4ebb7700 1 -- 172.21.15.42:0/1956425268 wait complete. 2023-12-12T09:57:26.064 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:26.400 INFO:teuthology.orchestra.run.smithi042.stdout:77309411387 2023-12-12T09:57:26.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.012+0000 7f17c189d700 1 -- 172.21.15.42:0/1040727389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17bc106570 msgr2=0x7f17bc106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.012+0000 7f17c189d700 1 --2- 172.21.15.42:0/1040727389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17bc106570 0x7f17bc106950 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0099c0 tx=0x7f17ac009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.012+0000 7f17c189d700 1 -- 172.21.15.42:0/1040727389 shutdown_connections 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.012+0000 7f17c189d700 1 --2- 172.21.15.42:0/1040727389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17bc06a620 0x7f17bc10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.012+0000 7f17c189d700 1 --2- 172.21.15.42:0/1040727389 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17bc106570 0x7f17bc106950 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.012+0000 7f17c189d700 1 --2- 172.21.15.42:0/1040727389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17bc069c60 0x7f17bc06a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.012+0000 7f17c189d700 1 -- 172.21.15.42:0/1040727389 >> 172.21.15.42:0/1040727389 conn(0x7f17bc0765b0 msgr2=0x7f17bc0769c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.012+0000 7f17c189d700 1 -- 172.21.15.42:0/1040727389 shutdown_connections 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.012+0000 7f17c189d700 1 -- 172.21.15.42:0/1040727389 wait complete. 2023-12-12T09:57:26.663 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.013+0000 7f17c189d700 1 Processor -- start 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.013+0000 7f17c189d700 1 -- start start 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17c189d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17bc069c60 0x7f17bc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17c189d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17bc06a620 0x7f17bc19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17c189d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17bc106570 0x7f17bc1977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17c189d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc197ea0 con 0x7f17bc106570 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17c189d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc198010 con 0x7f17bc069c60 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17c189d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc1981d0 con 0x7f17bc06a620 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17bc106570 0x7f17bc1977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17bc106570 0x7f17bc1977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34152/0 (socket says 172.21.15.42:34152) 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17bb7fe700 1 -- 172.21.15.42:0/2828539869 learned_addr learned my addr 172.21.15.42:0/2828539869 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17baffd700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17bc069c60 0x7f17bc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17ba7fc700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17bc06a620 0x7f17bc19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17bb7fe700 1 -- 172.21.15.42:0/2828539869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17bc06a620 msgr2=0x7f17bc19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:26.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17bb7fe700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17bc06a620 0x7f17bc19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17bb7fe700 1 -- 172.21.15.42:0/2828539869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17bc069c60 msgr2=0x7f17bc19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17bb7fe700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f17bc069c60 0x7f17bc19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17bb7fe700 1 -- 172.21.15.42:0/2828539869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac0095f0 con 0x7f17bc106570 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17ba7fc700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f17bc06a620 0x7f17bc19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17bb7fe700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17bc106570 0x7f17bc1977f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f17b400ada0 tx=0x7f17b400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.014+0000 7f17c089b700 1 -- 172.21.15.42:0/2828539869 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b400f070 con 0x7f17bc106570 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.015+0000 7f17c089b700 1 -- 172.21.15.42:0/2828539869 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b4007b30 con 0x7f17bc106570 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.015+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17bc1984b0 con 0x7f17bc106570 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.015+0000 7f17c089b700 1 -- 172.21.15.42:0/2828539869 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b40053e0 con 0x7f17bc106570 2023-12-12T09:57:26.665 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.015+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17bc1a4670 con 0x7f17bc106570 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.015+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f17bc111790 con 0x7f17bc106570 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.017+0000 7f17c089b700 1 -- 172.21.15.42:0/2828539869 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f17b400c980 con 0x7f17bc106570 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.017+0000 7f17c089b700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f17a80a1510 0x7f17a80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.018+0000 7f17baffd700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f17a80a1510 0x7f17a80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.018+0000 7f17c089b700 1 -- 172.21.15.42:0/2828539869 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f17b40cd040 con 0x7f17bc106570 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.018+0000 7f17c089b700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] conn(0x7f17a80ac090 0x7f17a80ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.018+0000 7f17c089b700 1 -- 172.21.15.42:0/2828539869 --> [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17a80aeba0 con 0x7f17a80ac090 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.018+0000 7f17c089b700 1 -- 172.21.15.42:0/2828539869 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17b40cd450 con 0x7f17bc106570 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.018+0000 7f17ba7fc700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] conn(0x7f17a80ac090 0x7f17a80ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.018+0000 7f17baffd700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f17a80a1510 0x7f17a80a39d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17ac006010 tx=0x7f17ac005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:26.666 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.018+0000 7f17ba7fc700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] conn(0x7f17a80ac090 0x7f17a80ae4d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.019+0000 7f17c089b700 1 -- 172.21.15.42:0/2828539869 <== osd.3 v2:172.21.15.42:6810/1515738461 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f17a80aeba0 con 0x7f17a80ac090 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.037+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 --> [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f17bc102c70 con 0x7f17a80ac090 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c089b700 1 -- 172.21.15.42:0/2828539869 <== osd.3 v2:172.21.15.42:6810/1515738461 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f17bc102c70 con 0x7f17a80ac090 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] conn(0x7f17a80ac090 msgr2=0x7f17a80ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] conn(0x7f17a80ac090 0x7f17a80ae4d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f17a80a1510 msgr2=0x7f17a80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f17a80a1510 0x7f17a80a39d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f17ac006010 tx=0x7f17ac005c40 comp rx=0 tx=0).stop 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17bc106570 msgr2=0x7f17bc1977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 --2- 172.21.15.42:0/2828539869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17bc106570 0x7f17bc1977f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f17b400ada0 tx=0x7f17b400c330 comp rx=0 tx=0).stop 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17bb7fe700 1 -- 172.21.15.42:0/2828539869 reap_dead start 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 shutdown_connections 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 >> 172.21.15.42:0/2828539869 conn(0x7f17bc0765b0 msgr2=0x7f17bc0fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 shutdown_connections 2023-12-12T09:57:26.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:26.038+0000 7f17c189d700 1 -- 172.21.15.42:0/2828539869 wait complete. 2023-12-12T09:57:26.853 INFO:teuthology.orchestra.run.smithi042.stdout:137438953523 2023-12-12T09:57:26.854 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-12-12T09:57:27.324 INFO:teuthology.orchestra.run.smithi042.stdout:279172874270 2023-12-12T09:57:27.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-12-12T09:57:27.565 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:27 smithi042 ceph-mon[98894]: pgmap v286: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-12-12T09:57:27.705 INFO:teuthology.orchestra.run.smithi042.stdout:244813135906 2023-12-12T09:57:27.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-12-12T09:57:27.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:27 smithi111 ceph-mon[102565]: pgmap v286: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-12-12T09:57:27.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:27 smithi173 ceph-mon[103308]: pgmap v286: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2023-12-12T09:57:27.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.817+0000 7f6f7226d700 1 -- 172.21.15.42:0/3656311988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c10ded0 msgr2=0x7f6f6c110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:27.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.817+0000 7f6f7226d700 1 --2- 172.21.15.42:0/3656311988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c10ded0 0x7f6f6c110340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400b0a0 tx=0x7f6f6400b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:27.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.817+0000 7f6f7226d700 1 -- 172.21.15.42:0/3656311988 shutdown_connections 2023-12-12T09:57:27.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.817+0000 7f6f7226d700 1 --2- 172.21.15.42:0/3656311988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c10ded0 0x7f6f6c110340 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:27.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.817+0000 7f6f7226d700 1 --2- 172.21.15.42:0/3656311988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f6c102570 0x7f6f6c069c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:27.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.817+0000 7f6f7226d700 1 --2- 172.21.15.42:0/3656311988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c06a1a0 0x7f6f6c06a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:27.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.817+0000 7f6f7226d700 1 -- 172.21.15.42:0/3656311988 >> 172.21.15.42:0/3656311988 conn(0x7f6f6c0772f0 msgr2=0x7f6f6c077700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:27.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.817+0000 7f6f7226d700 1 -- 172.21.15.42:0/3656311988 shutdown_connections 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.817+0000 7f6f7226d700 1 -- 172.21.15.42:0/3656311988 wait complete. 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.818+0000 7f6f7226d700 1 Processor -- start 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.818+0000 7f6f7226d700 1 -- start start 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.818+0000 7f6f7226d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f6c06a1a0 0x7f6f6c19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.818+0000 7f6f7226d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c102570 0x7f6f6c19ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.818+0000 7f6f7226d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c10ded0 0x7f6f6c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.818+0000 7f6f7226d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c19a200 con 0x7f6f6c10ded0 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.818+0000 7f6f7226d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c19a370 con 0x7f6f6c102570 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.818+0000 7f6f7226d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c19a530 con 0x7f6f6c06a1a0 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f7080a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c10ded0 0x7f6f6c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f6bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f6c06a1a0 0x7f6f6c19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f6bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f6c06a1a0 0x7f6f6c19fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:49146/0 (socket says 172.21.15.42:49146) 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f7080a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c10ded0 0x7f6f6c199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34164/0 (socket says 172.21.15.42:34164) 2023-12-12T09:57:27.919 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f6bfff700 1 -- 172.21.15.42:0/3372729525 learned_addr learned my addr 172.21.15.42:0/3372729525 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:27.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f6b7fe700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c102570 0x7f6f6c19ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:27.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f7080a700 1 -- 172.21.15.42:0/3372729525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f6c06a1a0 msgr2=0x7f6f6c19fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:27.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f7080a700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f6c06a1a0 0x7f6f6c19fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:27.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f7080a700 1 -- 172.21.15.42:0/3372729525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c102570 msgr2=0x7f6f6c19ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:27.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f7080a700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c102570 0x7f6f6c19ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f7080a700 1 -- 172.21.15.42:0/3372729525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64009d00 con 0x7f6f6c10ded0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f6b7fe700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f6c102570 0x7f6f6c19ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f7080a700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c10ded0 0x7f6f6c199ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400b070 tx=0x7f6f64007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f697fa700 1 -- 172.21.15.42:0/3372729525 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400e040 con 0x7f6f6c10ded0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.819+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6c19a7b0 con 0x7f6f6c10ded0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.820+0000 7f6f697fa700 1 -- 172.21.15.42:0/3372729525 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f6401d470 con 0x7f6f6c10ded0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.820+0000 7f6f697fa700 1 -- 172.21.15.42:0/3372729525 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f640052a0 con 0x7f6f6c10ded0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.820+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c1a6a50 con 0x7f6f6c10ded0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.820+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f6c113b60 con 0x7f6f6c10ded0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.822+0000 7f6f697fa700 1 -- 172.21.15.42:0/3372729525 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f6f6401d5e0 con 0x7f6f6c10ded0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.823+0000 7f6f697fa700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f540a1510 0x7f6f540a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.823+0000 7f6f6bfff700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f540a1510 0x7f6f540a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.823+0000 7f6f697fa700 1 -- 172.21.15.42:0/3372729525 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6f640ce6f0 con 0x7f6f6c10ded0 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.823+0000 7f6f697fa700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] conn(0x7f6f540ac090 0x7f6f540ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:27.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.823+0000 7f6f697fa700 1 -- 172.21.15.42:0/3372729525 --> [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6f540aeba0 con 0x7f6f540ac090 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.823+0000 7f6f697fa700 1 -- 172.21.15.42:0/3372729525 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f640ceb00 con 0x7f6f6c10ded0 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.824+0000 7f6f6bfff700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f540a1510 0x7f6f540a39d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c005950 tx=0x7f6f5c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.824+0000 7f6f6b7fe700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] conn(0x7f6f540ac090 0x7f6f540ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.824+0000 7f6f6b7fe700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] conn(0x7f6f540ac090 0x7f6f540ae4d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.825+0000 7f6f697fa700 1 -- 172.21.15.42:0/3372729525 <== osd.7 v2:172.21.15.111:6816/1072744652 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f6f540aeba0 con 0x7f6f540ac090 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.842+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 --> [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f6c100b00 con 0x7f6f540ac090 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.843+0000 7f6f697fa700 1 -- 172.21.15.42:0/3372729525 <== osd.7 v2:172.21.15.111:6816/1072744652 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6f6c100b00 con 0x7f6f540ac090 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.843+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 >> [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] conn(0x7f6f540ac090 msgr2=0x7f6f540ae4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.843+0000 7f6f7226d700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] conn(0x7f6f540ac090 0x7f6f540ae4d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.844+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f540a1510 msgr2=0x7f6f540a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.844+0000 7f6f7226d700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f540a1510 0x7f6f540a39d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c005950 tx=0x7f6f5c00b500 comp rx=0 tx=0).stop 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.844+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c10ded0 msgr2=0x7f6f6c199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.844+0000 7f6f7226d700 1 --2- 172.21.15.42:0/3372729525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f6c10ded0 0x7f6f6c199ba0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6f6400b070 tx=0x7f6f64007930 comp rx=0 tx=0).stop 2023-12-12T09:57:27.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.844+0000 7f6f7080a700 1 -- 172.21.15.42:0/3372729525 reap_dead start 2023-12-12T09:57:27.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.844+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 shutdown_connections 2023-12-12T09:57:27.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.844+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 >> 172.21.15.42:0/3372729525 conn(0x7f6f6c0772f0 msgr2=0x7f6f6c0ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:27.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.844+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 shutdown_connections 2023-12-12T09:57:27.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:27.844+0000 7f6f7226d700 1 -- 172.21.15.42:0/3372729525 wait complete. 2023-12-12T09:57:28.114 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:28.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2023-12-12T09:57:28.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-12-12T09:57:28.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-12-12T09:57:28.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-12T09:57:28.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-12-12T09:57:28.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2023-12-12T09:57:28.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-12-12T09:57:28.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]: dispatch 2023-12-12T09:57:28.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-12-12T09:57:28.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2023-12-12T09:57:28.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2023-12-12T09:57:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-12-12T09:57:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-12-12T09:57:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-12T09:57:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-12-12T09:57:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2023-12-12T09:57:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-12-12T09:57:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]: dispatch 2023-12-12T09:57:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-12-12T09:57:28.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2023-12-12T09:57:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2023-12-12T09:57:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2023-12-12T09:57:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2023-12-12T09:57:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-12T09:57:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2023-12-12T09:57:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2023-12-12T09:57:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-12-12T09:57:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]: dispatch 2023-12-12T09:57:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2023-12-12T09:57:28.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.103+0000 7f3aea9b5700 1 -- 172.21.15.42:0/3304522732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ae41076f0 msgr2=0x7f3ae4107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.103+0000 7f3aea9b5700 1 --2- 172.21.15.42:0/3304522732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ae41076f0 0x7f3ae4107ad0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4009990 tx=0x7f3ad4009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.103+0000 7f3aea9b5700 1 -- 172.21.15.42:0/3304522732 shutdown_connections 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.103+0000 7f3aea9b5700 1 --2- 172.21.15.42:0/3304522732 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ae4102040 0x7f3ae410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.103+0000 7f3aea9b5700 1 --2- 172.21.15.42:0/3304522732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ae41076f0 0x7f3ae4107ad0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.103+0000 7f3aea9b5700 1 --2- 172.21.15.42:0/3304522732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ae4101680 0x7f3ae4101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.103+0000 7f3aea9b5700 1 -- 172.21.15.42:0/3304522732 >> 172.21.15.42:0/3304522732 conn(0x7f3ae40787c0 msgr2=0x7f3ae4078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.103+0000 7f3aea9b5700 1 -- 172.21.15.42:0/3304522732 shutdown_connections 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.103+0000 7f3aea9b5700 1 -- 172.21.15.42:0/3304522732 wait complete. 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3aea9b5700 1 Processor -- start 2023-12-12T09:57:29.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3aea9b5700 1 -- start start 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3aea9b5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ae4101680 0x7f3ae419d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3aea9b5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ae4102040 0x7f3ae419dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3aea9b5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ae41076f0 0x7f3ae41977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3aea9b5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae4197e60 con 0x7f3ae41076f0 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3aea9b5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae4197fd0 con 0x7f3ae4102040 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3aea9b5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae4198190 con 0x7f3ae4101680 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3ae8f52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ae41076f0 0x7f3ae41977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3ae8f52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ae41076f0 0x7f3ae41977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34182/0 (socket says 172.21.15.42:34182) 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3ae8f52700 1 -- 172.21.15.42:0/75584837 learned_addr learned my addr 172.21.15.42:0/75584837 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3ae3fff700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ae4101680 0x7f3ae419d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.104+0000 7f3ae37fe700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ae4102040 0x7f3ae419dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3ae8f52700 1 -- 172.21.15.42:0/75584837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ae4101680 msgr2=0x7f3ae419d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3ae8f52700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ae4101680 0x7f3ae419d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3ae8f52700 1 -- 172.21.15.42:0/75584837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ae4102040 msgr2=0x7f3ae419dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3ae8f52700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ae4102040 0x7f3ae419dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3ae8f52700 1 -- 172.21.15.42:0/75584837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad40095f0 con 0x7f3ae41076f0 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3ae8f52700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ae41076f0 0x7f3ae41977f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00ab50 tx=0x7f3adc00ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3ae17fa700 1 -- 172.21.15.42:0/75584837 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc00f070 con 0x7f3ae41076f0 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae4198470 con 0x7f3ae41076f0 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3ae17fa700 1 -- 172.21.15.42:0/75584837 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3adc004bb0 con 0x7f3ae41076f0 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3ae17fa700 1 -- 172.21.15.42:0/75584837 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3adc005390 con 0x7f3ae41076f0 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.105+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae40739d0 con 0x7f3ae41076f0 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.106+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ad0000fc0 con 0x7f3ae41076f0 2023-12-12T09:57:29.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.108+0000 7f3ae17fa700 1 -- 172.21.15.42:0/75584837 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f3adc004d20 con 0x7f3ae41076f0 2023-12-12T09:57:29.110 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.108+0000 7f3ae17fa700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3acc0a1510 0x7f3acc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.110 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.108+0000 7f3ae3fff700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3acc0a1510 0x7f3acc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.110 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.108+0000 7f3ae17fa700 1 -- 172.21.15.42:0/75584837 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3adc0cd330 con 0x7f3ae41076f0 2023-12-12T09:57:29.110 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.108+0000 7f3ae17fa700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] conn(0x7f3acc0ac530 0x7f3acc0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.110 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.108+0000 7f3ae17fa700 1 -- 172.21.15.42:0/75584837 --> [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3acc0af0a0 con 0x7f3acc0ac530 2023-12-12T09:57:29.111 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.108+0000 7f3ae17fa700 1 -- 172.21.15.42:0/75584837 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3adc0cd740 con 0x7f3ae41076f0 2023-12-12T09:57:29.111 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.109+0000 7f3ae3fff700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3acc0a1510 0x7f3acc0a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40098e0 tx=0x7f3ad4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:29.111 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.109+0000 7f3ae37fe700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] conn(0x7f3acc0ac530 0x7f3acc0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.111 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.109+0000 7f3ae37fe700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] conn(0x7f3acc0ac530 0x7f3acc0ae9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:29.112 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.110+0000 7f3ae17fa700 1 -- 172.21.15.42:0/75584837 <== osd.1 v2:172.21.15.111:6800/804818401 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3acc0af0a0 con 0x7f3acc0ac530 2023-12-12T09:57:29.142 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.140+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 --> [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ad0002d40 con 0x7f3acc0ac530 2023-12-12T09:57:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3ae17fa700 1 -- 172.21.15.42:0/75584837 <== osd.1 v2:172.21.15.111:6800/804818401 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3ad0002d40 con 0x7f3acc0ac530 2023-12-12T09:57:29.143 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 >> [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] conn(0x7f3acc0ac530 msgr2=0x7f3acc0ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3aea9b5700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] conn(0x7f3acc0ac530 0x7f3acc0ae9d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3acc0a1510 msgr2=0x7f3acc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3aea9b5700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f3acc0a1510 0x7f3acc0a39d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3ad40098e0 tx=0x7f3ad4005960 comp rx=0 tx=0).stop 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ae41076f0 msgr2=0x7f3ae41977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3aea9b5700 1 --2- 172.21.15.42:0/75584837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ae41076f0 0x7f3ae41977f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3adc00ab50 tx=0x7f3adc00ae60 comp rx=0 tx=0).stop 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3ae8f52700 1 -- 172.21.15.42:0/75584837 reap_dead start 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 shutdown_connections 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.141+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 >> 172.21.15.42:0/75584837 conn(0x7f3ae40787c0 msgr2=0x7f3ae41003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.142+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 shutdown_connections 2023-12-12T09:57:29.144 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.142+0000 7f3aea9b5700 1 -- 172.21.15.42:0/75584837 wait complete. 2023-12-12T09:57:29.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.204+0000 7f3957a5b700 1 -- 172.21.15.42:0/884398598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3950102040 msgr2=0x7f395010e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.204+0000 7f3957a5b700 1 --2- 172.21.15.42:0/884398598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3950102040 0x7f395010e400 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f394800b0d0 tx=0x7f394800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 -- 172.21.15.42:0/884398598 shutdown_connections 2023-12-12T09:57:29.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 --2- 172.21.15.42:0/884398598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3950102040 0x7f395010e400 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 --2- 172.21.15.42:0/884398598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3950107630 0x7f3950107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 --2- 172.21.15.42:0/884398598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3950101680 0x7f3950101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 -- 172.21.15.42:0/884398598 >> 172.21.15.42:0/884398598 conn(0x7f39500786f0 msgr2=0x7f3950078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:29.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 -- 172.21.15.42:0/884398598 shutdown_connections 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 -- 172.21.15.42:0/884398598 wait complete. 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 Processor -- start 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 -- start start 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3950101680 0x7f395010b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3950102040 0x7f395010b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.205+0000 7f3957a5b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3950107630 0x7f39501a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3957a5b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39501a2ee0 con 0x7f3950102040 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3957a5b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39501a3050 con 0x7f3950101680 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3957a5b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39501a3210 con 0x7f3950107630 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3954ff6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3950102040 0x7f395010b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3954ff6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3950102040 0x7f395010b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34196/0 (socket says 172.21.15.42:34196) 2023-12-12T09:57:29.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3954ff6700 1 -- 172.21.15.42:0/3892526847 learned_addr learned my addr 172.21.15.42:0/3892526847 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3955ff8700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3950107630 0x7f39501a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f39557f7700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3950101680 0x7f395010b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3954ff6700 1 -- 172.21.15.42:0/3892526847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3950107630 msgr2=0x7f39501a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3954ff6700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3950107630 0x7f39501a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3954ff6700 1 -- 172.21.15.42:0/3892526847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3950101680 msgr2=0x7f395010b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3954ff6700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3950101680 0x7f395010b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3954ff6700 1 -- 172.21.15.42:0/3892526847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3948009d00 con 0x7f3950102040 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f39557f7700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3950101680 0x7f395010b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3954ff6700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3950102040 0x7f395010b980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f394400c9b0 tx=0x7f394400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f39427fc700 1 -- 172.21.15.42:0/3892526847 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944007b00 con 0x7f3950102040 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f39427fc700 1 -- 172.21.15.42:0/3892526847 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3944007c60 con 0x7f3950102040 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39501a34f0 con 0x7f3950102040 2023-12-12T09:57:29.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f39427fc700 1 -- 172.21.15.42:0/3892526847 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944018660 con 0x7f3950102040 2023-12-12T09:57:29.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.206+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39501aad50 con 0x7f3950102040 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.207+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3950111710 con 0x7f3950102040 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.209+0000 7f39427fc700 1 -- 172.21.15.42:0/3892526847 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f394400f450 con 0x7f3950102040 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.209+0000 7f39427fc700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1590 0x7f393c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.210+0000 7f39557f7700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1590 0x7f393c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.210+0000 7f39427fc700 1 -- 172.21.15.42:0/3892526847 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f39440cd530 con 0x7f3950102040 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.210+0000 7f39427fc700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] conn(0x7f393c0ac5b0 0x7f393c0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.210+0000 7f39427fc700 1 -- 172.21.15.42:0/3892526847 --> [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f393c0af120 con 0x7f393c0ac5b0 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.210+0000 7f39427fc700 1 -- 172.21.15.42:0/3892526847 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39440cd940 con 0x7f3950102040 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.210+0000 7f3955ff8700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] conn(0x7f393c0ac5b0 0x7f393c0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.210+0000 7f39557f7700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1590 0x7f393c0a3a50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f394c009cb0 tx=0x7f394c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:29.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.210+0000 7f3955ff8700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] conn(0x7f393c0ac5b0 0x7f393c0aea50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:29.214 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.211+0000 7f39427fc700 1 -- 172.21.15.42:0/3892526847 <== osd.6 v2:172.21.15.42:6818/288642128 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f393c0af120 con 0x7f393c0ac5b0 2023-12-12T09:57:29.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.229+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 --> [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f395010e2e0 con 0x7f393c0ac5b0 2023-12-12T09:57:29.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.230+0000 7f39427fc700 1 -- 172.21.15.42:0/3892526847 <== osd.6 v2:172.21.15.42:6818/288642128 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f395010e2e0 con 0x7f393c0ac5b0 2023-12-12T09:57:29.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] conn(0x7f393c0ac5b0 msgr2=0x7f393c0aea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] conn(0x7f393c0ac5b0 0x7f393c0aea50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1590 msgr2=0x7f393c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f393c0a1590 0x7f393c0a3a50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f394c009cb0 tx=0x7f394c009260 comp rx=0 tx=0).stop 2023-12-12T09:57:29.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3950102040 msgr2=0x7f395010b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:29.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 --2- 172.21.15.42:0/3892526847 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3950102040 0x7f395010b980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f394400c9b0 tx=0x7f394400ccc0 comp rx=0 tx=0).stop 2023-12-12T09:57:29.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3955ff8700 1 -- 172.21.15.42:0/3892526847 reap_dead start 2023-12-12T09:57:29.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 shutdown_connections 2023-12-12T09:57:29.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 >> 172.21.15.42:0/3892526847 conn(0x7f39500786f0 msgr2=0x7f395010c980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:29.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 shutdown_connections 2023-12-12T09:57:29.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:29.231+0000 7f3957a5b700 1 -- 172.21.15.42:0/3892526847 wait complete. 2023-12-12T09:57:29.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: pgmap v287: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2023-12-12T09:57:29.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2023-12-12T09:57:29.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-12-12T09:57:29.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-12-12T09:57:29.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-12T09:57:29.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-12-12T09:57:29.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2023-12-12T09:57:29.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-12-12T09:57:29.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]': finished 2023-12-12T09:57:29.531 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-12-12T09:57:29.531 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2023-12-12T09:57:29.531 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:29 smithi042 ceph-mon[98894]: osdmap e88: 12 total, 12 up, 12 in 2023-12-12T09:57:29.608 INFO:teuthology.orchestra.run.smithi042.stdout:107374182455 2023-12-12T09:57:29.609 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-12-12T09:57:29.612 INFO:teuthology.orchestra.run.smithi042.stdout:214748364838 2023-12-12T09:57:29.612 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-12-12T09:57:29.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: pgmap v287: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2023-12-12T09:57:29.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2023-12-12T09:57:29.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:29 smithi111 ceph-mon[102565]: osdmap e88: 12 total, 12 up, 12 in 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: pgmap v287: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 10 op/s 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [5, 10]}]': finished 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2023-12-12T09:57:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2023-12-12T09:57:29.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:29 smithi173 ceph-mon[103308]: osdmap e88: 12 total, 12 up, 12 in 2023-12-12T09:57:30.103 INFO:teuthology.orchestra.run.smithi042.stdout:55834574913 2023-12-12T09:57:30.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-12-12T09:57:30.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:30 smithi042 ceph-mon[98894]: osdmap e89: 12 total, 12 up, 12 in 2023-12-12T09:57:30.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.535+0000 7f40492c6700 1 -- 172.21.15.42:0/3263110915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4044105470 msgr2=0x7f4044105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:30.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.535+0000 7f40492c6700 1 --2- 172.21.15.42:0/3263110915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4044105470 0x7f4044105850 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f40340099c0 tx=0x7f4034009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:30.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.535+0000 7f40492c6700 1 -- 172.21.15.42:0/3263110915 shutdown_connections 2023-12-12T09:57:30.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.535+0000 7f40492c6700 1 --2- 172.21.15.42:0/3263110915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4044100e80 0x7f404410d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:30.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.535+0000 7f40492c6700 1 --2- 172.21.15.42:0/3263110915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4044105470 0x7f4044105850 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:30.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.535+0000 7f40492c6700 1 --2- 172.21.15.42:0/3263110915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40441004c0 0x7f4044100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:30.538 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.535+0000 7f40492c6700 1 -- 172.21.15.42:0/3263110915 >> 172.21.15.42:0/3263110915 conn(0x7f40440774a0 msgr2=0x7f40440778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:30.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.535+0000 7f40492c6700 1 -- 172.21.15.42:0/3263110915 shutdown_connections 2023-12-12T09:57:30.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.535+0000 7f40492c6700 1 -- 172.21.15.42:0/3263110915 wait complete. 2023-12-12T09:57:30.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.536+0000 7f40492c6700 1 Processor -- start 2023-12-12T09:57:30.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.536+0000 7f40492c6700 1 -- start start 2023-12-12T09:57:30.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.536+0000 7f40492c6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40441004c0 0x7f404410dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.536+0000 7f40492c6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4044100e80 0x7f404410e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.536+0000 7f40492c6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4044105470 0x7f404410e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.536+0000 7f40492c6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40441a5d40 con 0x7f40441004c0 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.536+0000 7f40492c6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40441a5eb0 con 0x7f4044105470 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.536+0000 7f40492c6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40441a6070 con 0x7f4044100e80 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f4042ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40441004c0 0x7f404410dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f4042ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40441004c0 0x7f404410dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34210/0 (socket says 172.21.15.42:34210) 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f4042ffd700 1 -- 172.21.15.42:0/2142044994 learned_addr learned my addr 172.21.15.42:0/2142044994 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f40427fc700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4044100e80 0x7f404410e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f40437fe700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4044105470 0x7f404410e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f4042ffd700 1 -- 172.21.15.42:0/2142044994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4044100e80 msgr2=0x7f404410e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f4042ffd700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4044100e80 0x7f404410e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:30.541 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f4042ffd700 1 -- 172.21.15.42:0/2142044994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4044105470 msgr2=0x7f404410e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f4042ffd700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4044105470 0x7f404410e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f4042ffd700 1 -- 172.21.15.42:0/2142044994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40340095f0 con 0x7f40441004c0 2023-12-12T09:57:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f4042ffd700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40441004c0 0x7f404410dd60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4034006010 tx=0x7f4034004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f402bfff700 1 -- 172.21.15.42:0/2142044994 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403401d070 con 0x7f40441004c0 2023-12-12T09:57:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f402bfff700 1 -- 172.21.15.42:0/2142044994 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f403400fb30 con 0x7f40441004c0 2023-12-12T09:57:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40441a62f0 con 0x7f40441004c0 2023-12-12T09:57:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f402bfff700 1 -- 172.21.15.42:0/2142044994 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034022c70 con 0x7f40441004c0 2023-12-12T09:57:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.537+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40441a6700 con 0x7f40441004c0 2023-12-12T09:57:30.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.538+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4044111800 con 0x7f40441004c0 2023-12-12T09:57:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.539+0000 7f402bfff700 1 -- 172.21.15.42:0/2142044994 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f403400fca0 con 0x7f40441004c0 2023-12-12T09:57:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.540+0000 7f402bfff700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f402c0a1510 0x7f402c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.540+0000 7f402bfff700 1 -- 172.21.15.42:0/2142044994 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f40340ced40 con 0x7f40441004c0 2023-12-12T09:57:30.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.540+0000 7f40427fc700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f402c0a1510 0x7f402c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:30.544 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.540+0000 7f402bfff700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] conn(0x7f402c0ac530 0x7f402c0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:30.544 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.540+0000 7f402bfff700 1 -- 172.21.15.42:0/2142044994 --> [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f402c0af0a0 con 0x7f402c0ac530 2023-12-12T09:57:30.544 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.540+0000 7f402bfff700 1 -- 172.21.15.42:0/2142044994 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40340cf150 con 0x7f40441004c0 2023-12-12T09:57:30.544 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.540+0000 7f40437fe700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] conn(0x7f402c0ac530 0x7f402c0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:30.544 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.541+0000 7f40427fc700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f402c0a1510 0x7f402c0a39d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f404410f480 tx=0x7f403800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:30.544 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.541+0000 7f40437fe700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] conn(0x7f402c0ac530 0x7f402c0ae9d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:30.544 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.541+0000 7f402bfff700 1 -- 172.21.15.42:0/2142044994 <== osd.0 v2:172.21.15.42:6802/4116302283 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f402c0af0a0 con 0x7f402c0ac530 2023-12-12T09:57:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.558+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 --> [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4044061cd0 con 0x7f402c0ac530 2023-12-12T09:57:30.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f402bfff700 1 -- 172.21.15.42:0/2142044994 <== osd.0 v2:172.21.15.42:6802/4116302283 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4044061cd0 con 0x7f402c0ac530 2023-12-12T09:57:30.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] conn(0x7f402c0ac530 msgr2=0x7f402c0ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:30.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] conn(0x7f402c0ac530 0x7f402c0ae9d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:30.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f402c0a1510 msgr2=0x7f402c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f402c0a1510 0x7f402c0a39d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f404410f480 tx=0x7f403800b410 comp rx=0 tx=0).stop 2023-12-12T09:57:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40441004c0 msgr2=0x7f404410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 --2- 172.21.15.42:0/2142044994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f40441004c0 0x7f404410dd60 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4034006010 tx=0x7f4034004730 comp rx=0 tx=0).stop 2023-12-12T09:57:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40437fe700 1 -- 172.21.15.42:0/2142044994 reap_dead start 2023-12-12T09:57:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 shutdown_connections 2023-12-12T09:57:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 >> 172.21.15.42:0/2142044994 conn(0x7f40440774a0 msgr2=0x7f40440fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 shutdown_connections 2023-12-12T09:57:30.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:30.559+0000 7f40492c6700 1 -- 172.21.15.42:0/2142044994 wait complete. 2023-12-12T09:57:30.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:30 smithi111 ceph-mon[102565]: osdmap e89: 12 total, 12 up, 12 in 2023-12-12T09:57:30.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:30 smithi173 ceph-mon[103308]: osdmap e89: 12 total, 12 up, 12 in 2023-12-12T09:57:31.063 INFO:teuthology.orchestra.run.smithi042.stdout:193273528363 2023-12-12T09:57:31.063 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-12-12T09:57:31.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.031+0000 7f58f43ee700 1 -- 172.21.15.42:0/240266409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58ec106510 msgr2=0x7f58ec1068f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.031+0000 7f58f43ee700 1 --2- 172.21.15.42:0/240266409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58ec106510 0x7f58ec1068f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009990 tx=0x7f58e8009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.032+0000 7f58f43ee700 1 -- 172.21.15.42:0/240266409 shutdown_connections 2023-12-12T09:57:31.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.032+0000 7f58f43ee700 1 --2- 172.21.15.42:0/240266409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ec06a1a0 0x7f58ec10e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.032+0000 7f58f43ee700 1 --2- 172.21.15.42:0/240266409 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58ec106510 0x7f58ec1068f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.032+0000 7f58f43ee700 1 --2- 172.21.15.42:0/240266409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58ec069800 0x7f58ec069c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.032+0000 7f58f43ee700 1 -- 172.21.15.42:0/240266409 >> 172.21.15.42:0/240266409 conn(0x7f58ec0774a0 msgr2=0x7f58ec0778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.032+0000 7f58f43ee700 1 -- 172.21.15.42:0/240266409 shutdown_connections 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.032+0000 7f58f43ee700 1 -- 172.21.15.42:0/240266409 wait complete. 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.032+0000 7f58f43ee700 1 Processor -- start 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.032+0000 7f58f43ee700 1 -- start start 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f43ee700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58ec069800 0x7f58ec19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f43ee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58ec06a1a0 0x7f58ec19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f43ee700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ec106510 0x7f58ec197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f43ee700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec197e70 con 0x7f58ec06a1a0 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f43ee700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec197fe0 con 0x7f58ec069800 2023-12-12T09:57:31.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f43ee700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec1981a0 con 0x7f58ec106510 2023-12-12T09:57:31.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f1989700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58ec06a1a0 0x7f58ec19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f1989700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58ec06a1a0 0x7f58ec19dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34228/0 (socket says 172.21.15.42:34228) 2023-12-12T09:57:31.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f1989700 1 -- 172.21.15.42:0/3499288774 learned_addr learned my addr 172.21.15.42:0/3499288774 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:31.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f218a700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58ec069800 0x7f58ec19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f298b700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ec106510 0x7f58ec197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f1989700 1 -- 172.21.15.42:0/3499288774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ec106510 msgr2=0x7f58ec197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f1989700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f58ec106510 0x7f58ec197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f1989700 1 -- 172.21.15.42:0/3499288774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58ec069800 msgr2=0x7f58ec19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f1989700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58ec069800 0x7f58ec19d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f1989700 1 -- 172.21.15.42:0/3499288774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e80095f0 con 0x7f58ec06a1a0 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f218a700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f58ec069800 0x7f58ec19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.033+0000 7f58f1989700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58ec06a1a0 0x7f58ec19dc00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00e9a0 tx=0x7f58dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.034+0000 7f58e37fe700 1 -- 172.21.15.42:0/3499288774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc009790 con 0x7f58ec06a1a0 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.034+0000 7f58e37fe700 1 -- 172.21.15.42:0/3499288774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58dc00cbd0 con 0x7f58ec06a1a0 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.034+0000 7f58e37fe700 1 -- 172.21.15.42:0/3499288774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc018680 con 0x7f58ec06a1a0 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.034+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ec198480 con 0x7f58ec06a1a0 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.034+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ec1a46b0 con 0x7f58ec06a1a0 2023-12-12T09:57:31.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.037+0000 7f58e37fe700 1 -- 172.21.15.42:0/3499288774 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f58dc00cd40 con 0x7f58ec06a1a0 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.037+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f58ec111780 con 0x7f58ec06a1a0 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.037+0000 7f58e37fe700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f58d80a1510 0x7f58d80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.037+0000 7f58e37fe700 1 -- 172.21.15.42:0/3499288774 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f58dc014070 con 0x7f58ec06a1a0 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.038+0000 7f58f218a700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f58d80a1510 0x7f58d80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.038+0000 7f58e37fe700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] conn(0x7f58d80ac530 0x7f58d80ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.038+0000 7f58e37fe700 1 -- 172.21.15.42:0/3499288774 --> [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f58d80af0a0 con 0x7f58d80ac530 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.038+0000 7f58e37fe700 1 -- 172.21.15.42:0/3499288774 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58dc0cf900 con 0x7f58ec06a1a0 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.038+0000 7f58f298b700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] conn(0x7f58d80ac530 0x7f58d80ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.038+0000 7f58f218a700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f58d80a1510 0x7f58d80a39d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b2e0 tx=0x7f58e8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.039+0000 7f58f298b700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] conn(0x7f58d80ac530 0x7f58d80ae9d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:31.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.040+0000 7f58e37fe700 1 -- 172.21.15.42:0/3499288774 <== osd.10 v2:172.21.15.111:6824/280900835 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f58d80af0a0 con 0x7f58d80ac530 2023-12-12T09:57:31.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.057+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 --> [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f58ec061cd0 con 0x7f58d80ac530 2023-12-12T09:57:31.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.057+0000 7f58e37fe700 1 -- 172.21.15.42:0/3499288774 <== osd.10 v2:172.21.15.111:6824/280900835 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f58ec061cd0 con 0x7f58d80ac530 2023-12-12T09:57:31.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 >> [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] conn(0x7f58d80ac530 msgr2=0x7f58d80ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] conn(0x7f58d80ac530 0x7f58d80ae9d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f58d80a1510 msgr2=0x7f58d80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f58d80a1510 0x7f58d80a39d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f58e800b2e0 tx=0x7f58e8005960 comp rx=0 tx=0).stop 2023-12-12T09:57:31.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58ec06a1a0 msgr2=0x7f58ec19dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 --2- 172.21.15.42:0/3499288774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f58ec06a1a0 0x7f58ec19dc00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00e9a0 tx=0x7f58dc00ecb0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f298b700 1 -- 172.21.15.42:0/3499288774 reap_dead start 2023-12-12T09:57:31.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 shutdown_connections 2023-12-12T09:57:31.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 >> 172.21.15.42:0/3499288774 conn(0x7f58ec0774a0 msgr2=0x7f58ec0fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:31.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 shutdown_connections 2023-12-12T09:57:31.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.058+0000 7f58f43ee700 1 -- 172.21.15.42:0/3499288774 wait complete. 2023-12-12T09:57:31.376 INFO:teuthology.orchestra.run.smithi042.stdout:34359738438 2023-12-12T09:57:31.377 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-12-12T09:57:31.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.626+0000 7f80764b2700 1 -- 172.21.15.42:0/1128943218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8070101680 msgr2=0x7f8070101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.630 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.626+0000 7f80764b2700 1 --2- 172.21.15.42:0/1128943218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8070101680 0x7f8070101b00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8064009990 tx=0x7f8064009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.630 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.628+0000 7f80764b2700 1 -- 172.21.15.42:0/1128943218 shutdown_connections 2023-12-12T09:57:31.630 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.628+0000 7f80764b2700 1 --2- 172.21.15.42:0/1128943218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8070102040 0x7f807010e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.630 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.628+0000 7f80764b2700 1 --2- 172.21.15.42:0/1128943218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80701076f0 0x7f8070107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.630 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.628+0000 7f80764b2700 1 --2- 172.21.15.42:0/1128943218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8070101680 0x7f8070101b00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.631 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.628+0000 7f80764b2700 1 -- 172.21.15.42:0/1128943218 >> 172.21.15.42:0/1128943218 conn(0x7f80700787c0 msgr2=0x7f8070078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:31.631 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.628+0000 7f80764b2700 1 -- 172.21.15.42:0/1128943218 shutdown_connections 2023-12-12T09:57:31.631 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.628+0000 7f80764b2700 1 -- 172.21.15.42:0/1128943218 wait complete. 2023-12-12T09:57:31.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.629+0000 7f80764b2700 1 Processor -- start 2023-12-12T09:57:31.632 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.629+0000 7f80764b2700 1 -- start start 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f80764b2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8070101680 0x7f80700738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f80764b2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8070102040 0x7f8070073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f80764b2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80701076f0 0x7f8070074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f80764b2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701a5d20 con 0x7f8070101680 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f80764b2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701a5e90 con 0x7f80701076f0 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f80764b2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80701a6050 con 0x7f8070102040 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8070101680 0x7f80700738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8070101680 0x7f80700738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34242/0 (socket says 172.21.15.42:34242) 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806ffff700 1 -- 172.21.15.42:0/426836055 learned_addr learned my addr 172.21.15.42:0/426836055 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f8074a4f700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80701076f0 0x7f8070074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806f7fe700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8070102040 0x7f8070073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806ffff700 1 -- 172.21.15.42:0/426836055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8070102040 msgr2=0x7f8070073e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806ffff700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8070102040 0x7f8070073e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.633 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806ffff700 1 -- 172.21.15.42:0/426836055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80701076f0 msgr2=0x7f8070074350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806ffff700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80701076f0 0x7f8070074350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806ffff700 1 -- 172.21.15.42:0/426836055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80640095f0 con 0x7f8070101680 2023-12-12T09:57:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806ffff700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8070101680 0x7f80700738d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f806000b530 tx=0x7f806000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.630+0000 7f806d7fa700 1 -- 172.21.15.42:0/426836055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060011830 con 0x7f8070101680 2023-12-12T09:57:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.631+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80701a6330 con 0x7f8070101680 2023-12-12T09:57:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.631+0000 7f806d7fa700 1 -- 172.21.15.42:0/426836055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8060011e50 con 0x7f8070101680 2023-12-12T09:57:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.631+0000 7f806d7fa700 1 -- 172.21.15.42:0/426836055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806000f2e0 con 0x7f8070101680 2023-12-12T09:57:31.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.631+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80701a6850 con 0x7f8070101680 2023-12-12T09:57:31.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.631+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8070111810 con 0x7f8070101680 2023-12-12T09:57:31.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.633+0000 7f806d7fa700 1 -- 172.21.15.42:0/426836055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f80600105a0 con 0x7f8070101680 2023-12-12T09:57:31.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.633+0000 7f806d7fa700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80580a1510 0x7f80580a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.633+0000 7f806f7fe700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80580a1510 0x7f80580a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.633+0000 7f806d7fa700 1 -- 172.21.15.42:0/426836055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f80600cd070 con 0x7f8070101680 2023-12-12T09:57:31.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.634+0000 7f806d7fa700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] conn(0x7f80580ac530 0x7f80580ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:31.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.634+0000 7f806d7fa700 1 -- 172.21.15.42:0/426836055 --> [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f80580af0a0 con 0x7f80580ac530 2023-12-12T09:57:31.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.634+0000 7f806d7fa700 1 -- 172.21.15.42:0/426836055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80600cd480 con 0x7f8070101680 2023-12-12T09:57:31.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.634+0000 7f806f7fe700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80580a1510 0x7f80580a39d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80640098e0 tx=0x7f8064005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:31.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.634+0000 7f8074a4f700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] conn(0x7f80580ac530 0x7f80580ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:31.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.634+0000 7f8074a4f700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] conn(0x7f80580ac530 0x7f80580ae9d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:31.637 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.635+0000 7f806d7fa700 1 -- 172.21.15.42:0/426836055 <== osd.5 v2:172.21.15.173:6808/3342915848 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f80580af0a0 con 0x7f80580ac530 2023-12-12T09:57:31.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:31 smithi042 ceph-mon[98894]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 11 op/s 2023-12-12T09:57:31.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.652+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 --> [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8070102d10 con 0x7f80580ac530 2023-12-12T09:57:31.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.652+0000 7f806d7fa700 1 -- 172.21.15.42:0/426836055 <== osd.5 v2:172.21.15.173:6808/3342915848 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8070102d10 con 0x7f80580ac530 2023-12-12T09:57:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 >> [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] conn(0x7f80580ac530 msgr2=0x7f80580ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] conn(0x7f80580ac530 0x7f80580ae9d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80580a1510 msgr2=0x7f80580a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f80580a1510 0x7f80580a39d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f80640098e0 tx=0x7f8064005960 comp rx=0 tx=0).stop 2023-12-12T09:57:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8070101680 msgr2=0x7f80700738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 --2- 172.21.15.42:0/426836055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8070101680 0x7f80700738d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f806000b530 tx=0x7f806000b840 comp rx=0 tx=0).stop 2023-12-12T09:57:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f8074a4f700 1 -- 172.21.15.42:0/426836055 reap_dead start 2023-12-12T09:57:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 shutdown_connections 2023-12-12T09:57:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 >> 172.21.15.42:0/426836055 conn(0x7f80700787c0 msgr2=0x7f8070100270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 shutdown_connections 2023-12-12T09:57:31.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:31.653+0000 7f80764b2700 1 -- 172.21.15.42:0/426836055 wait complete. 2023-12-12T09:57:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:31 smithi111 ceph-mon[102565]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 11 op/s 2023-12-12T09:57:31.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:31 smithi173 ceph-mon[103308]: pgmap v290: 129 pgs: 1 peering, 128 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 11 op/s 2023-12-12T09:57:32.054 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:32.141 INFO:teuthology.orchestra.run.smithi042.stdout:158913790000 2023-12-12T09:57:32.142 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-12-12T09:57:32.187 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:32.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.245+0000 7f478dc4a700 1 -- 172.21.15.42:0/2181771922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105470 msgr2=0x7f4788105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:32.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.245+0000 7f478dc4a700 1 --2- 172.21.15.42:0/2181771922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105470 0x7f4788105850 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f477800b0d0 tx=0x7f477800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:32.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.246+0000 7f478dc4a700 1 -- 172.21.15.42:0/2181771922 shutdown_connections 2023-12-12T09:57:32.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.246+0000 7f478dc4a700 1 --2- 172.21.15.42:0/2181771922 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4788100e80 0x7f478810d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:32.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.246+0000 7f478dc4a700 1 --2- 172.21.15.42:0/2181771922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4788105470 0x7f4788105850 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:32.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.246+0000 7f478dc4a700 1 --2- 172.21.15.42:0/2181771922 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f47881004c0 0x7f4788100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:32.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.246+0000 7f478dc4a700 1 -- 172.21.15.42:0/2181771922 >> 172.21.15.42:0/2181771922 conn(0x7f47880774a0 msgr2=0x7f47880778b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:32.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.246+0000 7f478dc4a700 1 -- 172.21.15.42:0/2181771922 shutdown_connections 2023-12-12T09:57:32.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.246+0000 7f478dc4a700 1 -- 172.21.15.42:0/2181771922 wait complete. 2023-12-12T09:57:32.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.246+0000 7f478dc4a700 1 Processor -- start 2023-12-12T09:57:32.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f478dc4a700 1 -- start start 2023-12-12T09:57:32.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f478dc4a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47881004c0 0x7f478819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:32.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f478dc4a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4788100e80 0x7f478819ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:32.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f478dc4a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4788105470 0x7f4788197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f478dc4a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47881980a0 con 0x7f47881004c0 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f478dc4a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788198210 con 0x7f4788105470 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f478dc4a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47881983d0 con 0x7f4788100e80 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f47877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47881004c0 0x7f478819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f47877fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47881004c0 0x7f478819d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34258/0 (socket says 172.21.15.42:34258) 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f47877fe700 1 -- 172.21.15.42:0/1122776786 learned_addr learned my addr 172.21.15.42:0/1122776786 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f4787fff700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4788105470 0x7f4788197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f4786ffd700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4788100e80 0x7f478819ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f47877fe700 1 -- 172.21.15.42:0/1122776786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4788100e80 msgr2=0x7f478819ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f47877fe700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4788100e80 0x7f478819ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f47877fe700 1 -- 172.21.15.42:0/1122776786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4788105470 msgr2=0x7f4788197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f47877fe700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4788105470 0x7f4788197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:32.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f47877fe700 1 -- 172.21.15.42:0/1122776786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778009d00 con 0x7f47881004c0 2023-12-12T09:57:32.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.247+0000 7f47877fe700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47881004c0 0x7f478819d8b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f477800bd50 tx=0x7f477800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:32.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.248+0000 7f4784ff9700 1 -- 172.21.15.42:0/1122776786 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477800e040 con 0x7f47881004c0 2023-12-12T09:57:32.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.248+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4788198650 con 0x7f47881004c0 2023-12-12T09:57:32.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.248+0000 7f4784ff9700 1 -- 172.21.15.42:0/1122776786 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f477801d930 con 0x7f47881004c0 2023-12-12T09:57:32.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.248+0000 7f4784ff9700 1 -- 172.21.15.42:0/1122776786 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477801b980 con 0x7f47881004c0 2023-12-12T09:57:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.248+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4788102200 con 0x7f47881004c0 2023-12-12T09:57:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.249+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4788110710 con 0x7f47881004c0 2023-12-12T09:57:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.250+0000 7f4784ff9700 1 -- 172.21.15.42:0/1122776786 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f477801d470 con 0x7f47881004c0 2023-12-12T09:57:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.251+0000 7f4784ff9700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f47700a1510 0x7f47700a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.251+0000 7f4786ffd700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f47700a1510 0x7f47700a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.251+0000 7f4784ff9700 1 -- 172.21.15.42:0/1122776786 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f47780cede0 con 0x7f47881004c0 2023-12-12T09:57:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.251+0000 7f4784ff9700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] conn(0x7f47700ac530 0x7f47700ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.251+0000 7f4784ff9700 1 -- 172.21.15.42:0/1122776786 --> [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47700af0a0 con 0x7f47700ac530 2023-12-12T09:57:32.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.251+0000 7f4784ff9700 1 -- 172.21.15.42:0/1122776786 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47780cf1f0 con 0x7f47881004c0 2023-12-12T09:57:32.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.251+0000 7f4786ffd700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f47700a1510 0x7f47700a39d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f477c005fd0 tx=0x7f477c005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:32.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.251+0000 7f4787fff700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] conn(0x7f47700ac530 0x7f47700ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:32.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.252+0000 7f4787fff700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] conn(0x7f47700ac530 0x7f47700ae9d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:32.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.253+0000 7f4784ff9700 1 -- 172.21.15.42:0/1122776786 <== osd.11 v2:172.21.15.173:6824/1783755959 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f47700af0a0 con 0x7f47700ac530 2023-12-12T09:57:32.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.270+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 --> [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4788061cd0 con 0x7f47700ac530 2023-12-12T09:57:32.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.271+0000 7f4784ff9700 1 -- 172.21.15.42:0/1122776786 <== osd.11 v2:172.21.15.173:6824/1783755959 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4788061cd0 con 0x7f47700ac530 2023-12-12T09:57:32.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 >> [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] conn(0x7f47700ac530 msgr2=0x7f47700ae9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:32.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] conn(0x7f47700ac530 0x7f47700ae9d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:32.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f47700a1510 msgr2=0x7f47700a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f47700a1510 0x7f47700a39d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f477c005fd0 tx=0x7f477c005f60 comp rx=0 tx=0).stop 2023-12-12T09:57:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47881004c0 msgr2=0x7f478819d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 --2- 172.21.15.42:0/1122776786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f47881004c0 0x7f478819d8b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f477800bd50 tx=0x7f477800bd80 comp rx=0 tx=0).stop 2023-12-12T09:57:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f4787fff700 1 -- 172.21.15.42:0/1122776786 reap_dead start 2023-12-12T09:57:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 shutdown_connections 2023-12-12T09:57:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 >> 172.21.15.42:0/1122776786 conn(0x7f47880774a0 msgr2=0x7f4788069c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 shutdown_connections 2023-12-12T09:57:32.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:32.272+0000 7f478dc4a700 1 -- 172.21.15.42:0/1122776786 wait complete. 2023-12-12T09:57:33.261 INFO:teuthology.orchestra.run.smithi042.stdout:300647710746 2023-12-12T09:57:33.261 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-12-12T09:57:33.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:33 smithi042 ceph-mon[98894]: pgmap v291: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 11 op/s; 142 B/s, 1 objects/s recovering 2023-12-12T09:57:33.414 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:33.431 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:33.603 INFO:teuthology.orchestra.run.smithi042.stdout:335007449110 2023-12-12T09:57:33.603 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-12-12T09:57:33.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:33 smithi111 ceph-mon[102565]: pgmap v291: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 11 op/s; 142 B/s, 1 objects/s recovering 2023-12-12T09:57:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:33 smithi173 ceph-mon[103308]: pgmap v291: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 511 B/s wr, 11 op/s; 142 B/s, 1 objects/s recovering 2023-12-12T09:57:34.311 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:34.597 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:34.613 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:35.383 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:35.510 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:35 smithi042 ceph-mon[98894]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 142 B/s, 1 objects/s recovering 2023-12-12T09:57:35.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:35 smithi111 ceph-mon[102565]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 142 B/s, 1 objects/s recovering 2023-12-12T09:57:35.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:35 smithi173 ceph-mon[103308]: pgmap v292: 129 pgs: 129 active+clean; 641 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 142 B/s, 1 objects/s recovering 2023-12-12T09:57:36.102 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:36.305 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:36.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.613+0000 7f5e8a09a700 1 -- 172.21.15.42:0/3750618278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84106560 msgr2=0x7f5e84106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:36.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.613+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/3750618278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84106560 0x7f5e84106940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5e740099c0 tx=0x7f5e74009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.613+0000 7f5e8a09a700 1 -- 172.21.15.42:0/3750618278 shutdown_connections 2023-12-12T09:57:36.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.613+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/3750618278 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84100ef0 0x7f5e8410d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.613+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/3750618278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84106560 0x7f5e84106940 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.613+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/3750618278 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e84100530 0x7f5e841009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.613+0000 7f5e8a09a700 1 -- 172.21.15.42:0/3750618278 >> 172.21.15.42:0/3750618278 conn(0x7f5e84076570 msgr2=0x7f5e84076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:36.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.613+0000 7f5e8a09a700 1 -- 172.21.15.42:0/3750618278 shutdown_connections 2023-12-12T09:57:36.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.614+0000 7f5e8a09a700 1 -- 172.21.15.42:0/3750618278 wait complete. 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.614+0000 7f5e8a09a700 1 Processor -- start 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.614+0000 7f5e8a09a700 1 -- start start 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e8a09a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e84100530 0x7f5e8419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e8a09a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84100ef0 0x7f5e8419fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e8a09a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84106560 0x7f5e84199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e8a09a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8419a110 con 0x7f5e84100ef0 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e8a09a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8419a280 con 0x7f5e84106560 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e8a09a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8419a440 con 0x7f5e84100530 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e82ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84100ef0 0x7f5e8419fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:36.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e82ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84100ef0 0x7f5e8419fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34276/0 (socket says 172.21.15.42:34276) 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e82ffd700 1 -- 172.21.15.42:0/2864588842 learned_addr learned my addr 172.21.15.42:0/2864588842 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e83fff700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84106560 0x7f5e84199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e837fe700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e84100530 0x7f5e8419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e82ffd700 1 -- 172.21.15.42:0/2864588842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e84100530 msgr2=0x7f5e8419f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e82ffd700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e84100530 0x7f5e8419f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e82ffd700 1 -- 172.21.15.42:0/2864588842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84106560 msgr2=0x7f5e84199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e82ffd700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84106560 0x7f5e84199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e82ffd700 1 -- 172.21.15.42:0/2864588842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740095f0 con 0x7f5e84100ef0 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.615+0000 7f5e82ffd700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84100ef0 0x7f5e8419fe60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800b530 tx=0x7f5e7800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.616+0000 7f5e80ff9700 1 -- 172.21.15.42:0/2864588842 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e78011830 con 0x7f5e84100ef0 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.616+0000 7f5e80ff9700 1 -- 172.21.15.42:0/2864588842 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e78011e50 con 0x7f5e84100ef0 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.616+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8419a720 con 0x7f5e84100ef0 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.616+0000 7f5e80ff9700 1 -- 172.21.15.42:0/2864588842 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800f2e0 con 0x7f5e84100ef0 2023-12-12T09:57:36.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.616+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e841a6910 con 0x7f5e84100ef0 2023-12-12T09:57:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.617+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e84108a10 con 0x7f5e84100ef0 2023-12-12T09:57:36.620 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.619+0000 7f5e80ff9700 1 -- 172.21.15.42:0/2864588842 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f5e78010680 con 0x7f5e84100ef0 2023-12-12T09:57:36.621 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.619+0000 7f5e80ff9700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e6c0a14f0 0x7f5e6c0a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:36.621 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.619+0000 7f5e837fe700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e6c0a14f0 0x7f5e6c0a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:36.621 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.620+0000 7f5e80ff9700 1 -- 172.21.15.42:0/2864588842 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5e780ccc00 con 0x7f5e84100ef0 2023-12-12T09:57:36.622 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.620+0000 7f5e837fe700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e6c0a14f0 0x7f5e6c0a39b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e74006010 tx=0x7f5e74000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:36.627 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.625+0000 7f5e80ff9700 1 -- 172.21.15.42:0/2864588842 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5e78095f00 con 0x7f5e84100ef0 2023-12-12T09:57:36.834 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.832+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5e84101bd0 con 0x7f5e84100ef0 2023-12-12T09:57:36.834 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.832+0000 7f5e80ff9700 1 -- 172.21.15.42:0/2864588842 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e78015090 con 0x7f5e84100ef0 2023-12-12T09:57:36.834 INFO:teuthology.orchestra.run.smithi042.stdout:137438953525 2023-12-12T09:57:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e6c0a14f0 msgr2=0x7f5e6c0a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e6c0a14f0 0x7f5e6c0a39b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e74006010 tx=0x7f5e74000bc0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84100ef0 msgr2=0x7f5e8419fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84100ef0 0x7f5e8419fe60 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5e7800b530 tx=0x7f5e7800b840 comp rx=0 tx=0).stop 2023-12-12T09:57:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 shutdown_connections 2023-12-12T09:57:36.836 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5e6c0a14f0 0x7f5e6c0a39b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84106560 0x7f5e84199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5e84100530 0x7f5e8419f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 --2- 172.21.15.42:0/2864588842 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e84100ef0 0x7f5e8419fe60 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 >> 172.21.15.42:0/2864588842 conn(0x7f5e84076570 msgr2=0x7f5e840ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 shutdown_connections 2023-12-12T09:57:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:36.834+0000 7f5e8a09a700 1 -- 172.21.15.42:0/2864588842 wait complete. 2023-12-12T09:57:37.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.388+0000 7f9ad0b82700 1 -- 172.21.15.42:0/2367676896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100530 msgr2=0x7f9acc1009b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.388+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/2367676896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100530 0x7f9acc1009b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9ac00099c0 tx=0x7f9ac0009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.388+0000 7f9ad0b82700 1 -- 172.21.15.42:0/2367676896 shutdown_connections 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.388+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/2367676896 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9acc106560 0x7f9acc106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.388+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/2367676896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc100ef0 0x7f9acc10d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.388+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/2367676896 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100530 0x7f9acc1009b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.388+0000 7f9ad0b82700 1 -- 172.21.15.42:0/2367676896 >> 172.21.15.42:0/2367676896 conn(0x7f9acc076570 msgr2=0x7f9acc076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.388+0000 7f9ad0b82700 1 -- 172.21.15.42:0/2367676896 shutdown_connections 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.388+0000 7f9ad0b82700 1 -- 172.21.15.42:0/2367676896 wait complete. 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ad0b82700 1 Processor -- start 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ad0b82700 1 -- start start 2023-12-12T09:57:37.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ad0b82700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc100530 0x7f9acc1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ad0b82700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100ef0 0x7f9acc198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ad0b82700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9acc106560 0x7f9acc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ad0b82700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a3ae0 con 0x7f9acc100ef0 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ad0b82700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a3c50 con 0x7f9acc106560 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ad0b82700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc1a3e10 con 0x7f9acc100530 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ac9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100ef0 0x7f9acc198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ac9d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100ef0 0x7f9acc198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34290/0 (socket says 172.21.15.42:34290) 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9ac9d9b700 1 -- 172.21.15.42:0/3364202633 learned_addr learned my addr 172.21.15.42:0/3364202633 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9aca59c700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc100530 0x7f9acc1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.389+0000 7f9acad9d700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9acc106560 0x7f9acc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9acad9d700 1 -- 172.21.15.42:0/3364202633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc100530 msgr2=0x7f9acc1987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9acad9d700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc100530 0x7f9acc1987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9acad9d700 1 -- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100ef0 msgr2=0x7f9acc198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9acad9d700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100ef0 0x7f9acc198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9acad9d700 1 -- 172.21.15.42:0/3364202633 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac00095f0 con 0x7f9acc106560 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9ac9d9b700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100ef0 0x7f9acc198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9aca59c700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc100530 0x7f9acc1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9acad9d700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9acc106560 0x7f9acc199290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4007f10 tx=0x7f9ac400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9abb7fe700 1 -- 172.21.15.42:0/3364202633 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4009ea0 con 0x7f9acc106560 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9abb7fe700 1 -- 172.21.15.42:0/3364202633 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ac400f040 con 0x7f9acc106560 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc1a40f0 con 0x7f9acc106560 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9abb7fe700 1 -- 172.21.15.42:0/3364202633 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4004400 con 0x7f9acc106560 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.390+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc1a45a0 con 0x7f9acc106560 2023-12-12T09:57:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.392+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc108a10 con 0x7f9acc106560 2023-12-12T09:57:37.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.393+0000 7f9abb7fe700 1 -- 172.21.15.42:0/3364202633 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9ac4004ad0 con 0x7f9acc106560 2023-12-12T09:57:37.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.393+0000 7f9abb7fe700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9ab40a1510 0x7f9ab40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:37.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.394+0000 7f9aca59c700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9ab40a1510 0x7f9ab40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:37.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.394+0000 7f9abb7fe700 1 -- 172.21.15.42:0/3364202633 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9ac40ccb70 con 0x7f9acc106560 2023-12-12T09:57:37.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.394+0000 7f9aca59c700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9ab40a1510 0x7f9ab40a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009c90 tx=0x7f9abc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:37.537 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.398+0000 7f9abb7fe700 1 -- 172.21.15.42:0/3364202633 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9ac4095d00 con 0x7f9acc106560 2023-12-12T09:57:37.609 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.607+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9acc101bd0 con 0x7f9acc106560 2023-12-12T09:57:37.609 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.607+0000 7f9abb7fe700 1 -- 172.21.15.42:0/3364202633 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9ac4019090 con 0x7f9acc106560 2023-12-12T09:57:37.610 INFO:teuthology.orchestra.run.smithi042.stdout:77309411389 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9ab40a1510 msgr2=0x7f9ab40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9ab40a1510 0x7f9ab40a39d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9abc009c90 tx=0x7f9abc009260 comp rx=0 tx=0).stop 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9acc106560 msgr2=0x7f9acc199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9acc106560 0x7f9acc199290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4007f10 tx=0x7f9ac400d3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 shutdown_connections 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9ab40a1510 0x7f9ab40a39d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9acc106560 0x7f9acc199290 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9acc100530 0x7f9acc1987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 --2- 172.21.15.42:0/3364202633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9acc100ef0 0x7f9acc198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.610+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 >> 172.21.15.42:0/3364202633 conn(0x7f9acc076570 msgr2=0x7f9acc0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:37.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.611+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 shutdown_connections 2023-12-12T09:57:37.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.611+0000 7f9ad0b82700 1 -- 172.21.15.42:0/3364202633 wait complete. 2023-12-12T09:57:37.650 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:37 smithi042 ceph-mon[98894]: pgmap v293: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 10 op/s; 142 B/s, 1 objects/s recovering 2023-12-12T09:57:37.650 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:37 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2864588842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.685+0000 7f8188a0c700 1 -- 172.21.15.42:0/2427106312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184100170 msgr2=0x7f8184100550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.685+0000 7f8188a0c700 1 --2- 172.21.15.42:0/2427106312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184100170 0x7f8184100550 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8174009990 tx=0x7f8174009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.685+0000 7f8188a0c700 1 -- 172.21.15.42:0/2427106312 shutdown_connections 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.685+0000 7f8188a0c700 1 --2- 172.21.15.42:0/2427106312 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8184106940 0x7f818410e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.685+0000 7f8188a0c700 1 --2- 172.21.15.42:0/2427106312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184100170 0x7f8184100550 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.685+0000 7f8188a0c700 1 --2- 172.21.15.42:0/2427106312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8184100b20 0x7f8184106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.685+0000 7f8188a0c700 1 -- 172.21.15.42:0/2427106312 >> 172.21.15.42:0/2427106312 conn(0x7f8184075830 msgr2=0x7f8184075c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.685+0000 7f8188a0c700 1 -- 172.21.15.42:0/2427106312 shutdown_connections 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.685+0000 7f8188a0c700 1 -- 172.21.15.42:0/2427106312 wait complete. 2023-12-12T09:57:37.688 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8188a0c700 1 Processor -- start 2023-12-12T09:57:37.689 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8188a0c700 1 -- start start 2023-12-12T09:57:37.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8188a0c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8184100170 0x7f8184197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:37.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8188a0c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8184100b20 0x7f8184197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8188a0c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184106940 0x7f818419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8188a0c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818419c900 con 0x7f8184106940 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8188a0c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818419ca70 con 0x7f8184100b20 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8188a0c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818419cc30 con 0x7f8184100170 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8182d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184106940 0x7f818419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8182d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184106940 0x7f818419c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34306/0 (socket says 172.21.15.42:34306) 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8182d9d700 1 -- 172.21.15.42:0/3320608890 learned_addr learned my addr 172.21.15.42:0/3320608890 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.686+0000 7f8179d9b700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8184100b20 0x7f8184197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f818259c700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8184100170 0x7f8184197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f8182d9d700 1 -- 172.21.15.42:0/3320608890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8184100170 msgr2=0x7f8184197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f8182d9d700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8184100170 0x7f8184197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f8182d9d700 1 -- 172.21.15.42:0/3320608890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8184100b20 msgr2=0x7f8184197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f8182d9d700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8184100b20 0x7f8184197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f8182d9d700 1 -- 172.21.15.42:0/3320608890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81740095f0 con 0x7f8184106940 2023-12-12T09:57:37.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f8182d9d700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184106940 0x7f818419c330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f817c007d70 tx=0x7f817c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f817bfff700 1 -- 172.21.15.42:0/3320608890 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c00ac90 con 0x7f8184106940 2023-12-12T09:57:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f818419cf10 con 0x7f8184106940 2023-12-12T09:57:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f817bfff700 1 -- 172.21.15.42:0/3320608890 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f817c00adf0 con 0x7f8184106940 2023-12-12T09:57:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f817bfff700 1 -- 172.21.15.42:0/3320608890 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f817c0115e0 con 0x7f8184106940 2023-12-12T09:57:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.687+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81841011f0 con 0x7f8184106940 2023-12-12T09:57:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.688+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8184109b80 con 0x7f8184106940 2023-12-12T09:57:37.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.690+0000 7f817bfff700 1 -- 172.21.15.42:0/3320608890 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f817c012420 con 0x7f8184106940 2023-12-12T09:57:37.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.691+0000 7f817bfff700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f81700a1510 0x7f81700a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:37.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.691+0000 7f817bfff700 1 -- 172.21.15.42:0/3320608890 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f817c0cd680 con 0x7f8184106940 2023-12-12T09:57:37.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.691+0000 7f818259c700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f81700a1510 0x7f81700a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:37.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.691+0000 7f818259c700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f81700a1510 0x7f81700a39d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f817400b5c0 tx=0x7f8174005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:37.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.694+0000 7f817bfff700 1 -- 172.21.15.42:0/3320608890 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f817c096920 con 0x7f8184106940 2023-12-12T09:57:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:37 smithi111 ceph-mon[102565]: pgmap v293: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 10 op/s; 142 B/s, 1 objects/s recovering 2023-12-12T09:57:37.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:37 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2864588842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-12T09:57:37.801 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953523 got 137438953525 for osd.4 2023-12-12T09:57:37.801 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:37.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:37 smithi173 ceph-mon[103308]: pgmap v293: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 383 B/s wr, 10 op/s; 142 B/s, 1 objects/s recovering 2023-12-12T09:57:37.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:37 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2864588842' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-12T09:57:37.930 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.928+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f81840697f0 con 0x7f8184106940 2023-12-12T09:57:37.930 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.928+0000 7f817bfff700 1 -- 172.21.15.42:0/3320608890 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f817c099f40 con 0x7f8184106940 2023-12-12T09:57:37.930 INFO:teuthology.orchestra.run.smithi042.stdout:244813135908 2023-12-12T09:57:37.932 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f81700a1510 msgr2=0x7f81700a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f81700a1510 0x7f81700a39d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f817400b5c0 tx=0x7f8174005960 comp rx=0 tx=0).stop 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184106940 msgr2=0x7f818419c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184106940 0x7f818419c330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f817c007d70 tx=0x7f817c00a340 comp rx=0 tx=0).stop 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 shutdown_connections 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f81700a1510 0x7f81700a39d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8184106940 0x7f818419c330 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8184100170 0x7f8184197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 --2- 172.21.15.42:0/3320608890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8184100b20 0x7f8184197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.930+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 >> 172.21.15.42:0/3320608890 conn(0x7f8184075830 msgr2=0x7f81840fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.931+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 shutdown_connections 2023-12-12T09:57:37.933 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:37.931+0000 7f8188a0c700 1 -- 172.21.15.42:0/3320608890 wait complete. 2023-12-12T09:57:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.009+0000 7f93d3234700 1 -- 172.21.15.42:0/299243999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 msgr2=0x7f93cc106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.009+0000 7f93d3234700 1 --2- 172.21.15.42:0/299243999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 0x7f93cc106940 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f93c00099c0 tx=0x7f93c0009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.009+0000 7f93d3234700 1 -- 172.21.15.42:0/299243999 shutdown_connections 2023-12-12T09:57:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.009+0000 7f93d3234700 1 --2- 172.21.15.42:0/299243999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93cc100ef0 0x7f93cc10d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.009+0000 7f93d3234700 1 --2- 172.21.15.42:0/299243999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 0x7f93cc106940 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.009+0000 7f93d3234700 1 --2- 172.21.15.42:0/299243999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc100530 0x7f93cc1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.009+0000 7f93d3234700 1 -- 172.21.15.42:0/299243999 >> 172.21.15.42:0/299243999 conn(0x7f93cc076570 msgr2=0x7f93cc076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.010+0000 7f93d3234700 1 -- 172.21.15.42:0/299243999 shutdown_connections 2023-12-12T09:57:38.013 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.010+0000 7f93d3234700 1 -- 172.21.15.42:0/299243999 wait complete. 2023-12-12T09:57:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.010+0000 7f93d3234700 1 Processor -- start 2023-12-12T09:57:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.010+0000 7f93d3234700 1 -- start start 2023-12-12T09:57:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d3234700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93cc100530 0x7f93cc19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d3234700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc100ef0 0x7f93cc19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d3234700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 0x7f93cc1966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d3234700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc196d10 con 0x7f93cc106560 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d3234700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc196e80 con 0x7f93cc100530 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d3234700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc197040 con 0x7f93cc100ef0 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d17d1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 0x7f93cc1966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d17d1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 0x7f93cc1966b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34318/0 (socket says 172.21.15.42:34318) 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d17d1700 1 -- 172.21.15.42:0/1028381178 learned_addr learned my addr 172.21.15.42:0/1028381178 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93cbfff700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc100ef0 0x7f93cc19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d0fd0700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93cc100530 0x7f93cc19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d17d1700 1 -- 172.21.15.42:0/1028381178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc100ef0 msgr2=0x7f93cc19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d17d1700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc100ef0 0x7f93cc19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d17d1700 1 -- 172.21.15.42:0/1028381178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93cc100530 msgr2=0x7f93cc19c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d17d1700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93cc100530 0x7f93cc19c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.011+0000 7f93d17d1700 1 -- 172.21.15.42:0/1028381178 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c00095f0 con 0x7f93cc106560 2023-12-12T09:57:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.012+0000 7f93d17d1700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 0x7f93cc1966b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f93c4007ee0 tx=0x7f93c400f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.012+0000 7f93c9ffb700 1 -- 172.21.15.42:0/1028381178 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c400fd30 con 0x7f93cc106560 2023-12-12T09:57:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.012+0000 7f93c9ffb700 1 -- 172.21.15.42:0/1028381178 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93c400fe90 con 0x7f93cc106560 2023-12-12T09:57:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.012+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93cc197320 con 0x7f93cc106560 2023-12-12T09:57:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.012+0000 7f93c9ffb700 1 -- 172.21.15.42:0/1028381178 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c4005020 con 0x7f93cc106560 2023-12-12T09:57:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.012+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93cc069140 con 0x7f93cc106560 2023-12-12T09:57:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.013+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93cc101bd0 con 0x7f93cc106560 2023-12-12T09:57:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.014+0000 7f93c9ffb700 1 -- 172.21.15.42:0/1028381178 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f93c4004c40 con 0x7f93cc106560 2023-12-12T09:57:38.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.015+0000 7f93c9ffb700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f93bc0a5920 0x7f93bc0a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:38.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.015+0000 7f93d0fd0700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f93bc0a5920 0x7f93bc0a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:38.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.015+0000 7f93c9ffb700 1 -- 172.21.15.42:0/1028381178 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f93c40cc7f0 con 0x7f93cc106560 2023-12-12T09:57:38.017 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.016+0000 7f93d0fd0700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f93bc0a5920 0x7f93bc0a7de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009990 tx=0x7f93c0003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:38.026 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.024+0000 7f93c9ffb700 1 -- 172.21.15.42:0/1028381178 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f93c40959e0 con 0x7f93cc106560 2023-12-12T09:57:38.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.232+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f93cc04f290 con 0x7f93cc106560 2023-12-12T09:57:38.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.233+0000 7f93c9ffb700 1 -- 172.21.15.42:0/1028381178 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93c4099000 con 0x7f93cc106560 2023-12-12T09:57:38.235 INFO:teuthology.orchestra.run.smithi042.stdout:279172874272 2023-12-12T09:57:38.237 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f93bc0a5920 msgr2=0x7f93bc0a7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:38.237 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f93bc0a5920 0x7f93bc0a7de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009990 tx=0x7f93c0003680 comp rx=0 tx=0).stop 2023-12-12T09:57:38.237 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 msgr2=0x7f93cc1966b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:38.237 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 0x7f93cc1966b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f93c4007ee0 tx=0x7f93c400f5c0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.237 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 shutdown_connections 2023-12-12T09:57:38.238 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f93bc0a5920 0x7f93bc0a7de0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.238 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93cc100530 0x7f93cc19c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.238 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f93cc106560 0x7f93cc1966b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.238 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 --2- 172.21.15.42:0/1028381178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93cc100ef0 0x7f93cc19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:38.238 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 >> 172.21.15.42:0/1028381178 conn(0x7f93cc076570 msgr2=0x7f93cc0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:38.238 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 shutdown_connections 2023-12-12T09:57:38.238 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:38.235+0000 7f93d3234700 1 -- 172.21.15.42:0/1028381178 wait complete. 2023-12-12T09:57:38.477 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411387 got 77309411389 for osd.2 2023-12-12T09:57:38.477 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:38 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3364202633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-12T09:57:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:38 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3320608890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-12T09:57:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:38 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1028381178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-12T09:57:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:38 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3364202633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-12T09:57:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:38 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3320608890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-12T09:57:38.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:38 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1028381178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-12T09:57:38.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:38 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3364202633' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-12T09:57:38.831 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:38 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3320608890' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-12T09:57:38.832 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:38 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1028381178' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-12T09:57:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:39 smithi042 ceph-mon[98894]: pgmap v294: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 324 B/s wr, 9 op/s; 120 B/s, 1 objects/s recovering 2023-12-12T09:57:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:39 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:57:39.675 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.672+0000 7f5104133700 1 -- 172.21.15.42:0/3033818722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc106560 msgr2=0x7f50fc106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:39.675 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.672+0000 7f5104133700 1 --2- 172.21.15.42:0/3033818722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc106560 0x7f50fc106940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f50f80099c0 tx=0x7f50f8009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.672+0000 7f5104133700 1 -- 172.21.15.42:0/3033818722 shutdown_connections 2023-12-12T09:57:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.672+0000 7f5104133700 1 --2- 172.21.15.42:0/3033818722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100ef0 0x7f50fc10d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.672+0000 7f5104133700 1 --2- 172.21.15.42:0/3033818722 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc106560 0x7f50fc106940 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f50f80099c0 tx=0x7f50f8009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.672+0000 7f5104133700 1 --2- 172.21.15.42:0/3033818722 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50fc100530 0x7f50fc1009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.672+0000 7f5104133700 1 -- 172.21.15.42:0/3033818722 >> 172.21.15.42:0/3033818722 conn(0x7f50fc076570 msgr2=0x7f50fc076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.673+0000 7f5104133700 1 -- 172.21.15.42:0/3033818722 shutdown_connections 2023-12-12T09:57:39.676 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.673+0000 7f5104133700 1 -- 172.21.15.42:0/3033818722 wait complete. 2023-12-12T09:57:39.677 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5104133700 1 Processor -- start 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5104133700 1 -- start start 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5104133700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc100530 0x7f50fc1093e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5104133700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100ef0 0x7f50fc109920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5104133700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50fc109e60 0x7f50fc1aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5104133700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc10d5d0 con 0x7f50fc100530 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5104133700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1aa8e0 con 0x7f50fc109e60 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5104133700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1aaaa0 con 0x7f50fc100ef0 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5101ecf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc100530 0x7f50fc1093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5101ecf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc100530 0x7f50fc1093e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34324/0 (socket says 172.21.15.42:34324) 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.674+0000 7f5101ecf700 1 -- 172.21.15.42:0/3449879657 learned_addr learned my addr 172.21.15.42:0/3449879657 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f51026d0700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50fc109e60 0x7f50fc1aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f51016ce700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100ef0 0x7f50fc109920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:39.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f5101ecf700 1 -- 172.21.15.42:0/3449879657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100ef0 msgr2=0x7f50fc109920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f5101ecf700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100ef0 0x7f50fc109920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f5101ecf700 1 -- 172.21.15.42:0/3449879657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50fc109e60 msgr2=0x7f50fc1aa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f5101ecf700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50fc109e60 0x7f50fc1aa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f5101ecf700 1 -- 172.21.15.42:0/3449879657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f80095f0 con 0x7f50fc100530 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f51026d0700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50fc109e60 0x7f50fc1aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f5101ecf700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc100530 0x7f50fc1093e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006010 tx=0x7f50f8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f50f2ffd700 1 -- 172.21.15.42:0/3449879657 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f801d070 con 0x7f50fc100530 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc1aad20 con 0x7f50fc100530 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f50f2ffd700 1 -- 172.21.15.42:0/3449879657 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50f80059e0 con 0x7f50fc100530 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f50f2ffd700 1 -- 172.21.15.42:0/3449879657 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f800f950 con 0x7f50fc100530 2023-12-12T09:57:39.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.675+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc1ab130 con 0x7f50fc100530 2023-12-12T09:57:39.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.676+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc101bd0 con 0x7f50fc100530 2023-12-12T09:57:39.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.678+0000 7f50f2ffd700 1 -- 172.21.15.42:0/3449879657 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f50f8004fa0 con 0x7f50fc100530 2023-12-12T09:57:39.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.679+0000 7f50f2ffd700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f50e80a1510 0x7f50e80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:39.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.679+0000 7f50f2ffd700 1 -- 172.21.15.42:0/3449879657 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50f80cdfe0 con 0x7f50fc100530 2023-12-12T09:57:39.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.680+0000 7f51016ce700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f50e80a1510 0x7f50e80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:39.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.680+0000 7f51016ce700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f50e80a1510 0x7f50e80a39d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f50ec005950 tx=0x7f50ec0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:39.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.683+0000 7f50f2ffd700 1 -- 172.21.15.42:0/3449879657 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f50f8097200 con 0x7f50fc100530 2023-12-12T09:57:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:39 smithi111 ceph-mon[102565]: pgmap v294: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 324 B/s wr, 9 op/s; 120 B/s, 1 objects/s recovering 2023-12-12T09:57:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:39 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:57:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:39 smithi173 ceph-mon[103308]: pgmap v294: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 324 B/s wr, 9 op/s; 120 B/s, 1 objects/s recovering 2023-12-12T09:57:39.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:39 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:57:39.898 INFO:teuthology.orchestra.run.smithi042.stdout:214748364840 2023-12-12T09:57:39.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.895+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f50fc04f290 con 0x7f50fc100530 2023-12-12T09:57:39.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.896+0000 7f50f2ffd700 1 -- 172.21.15.42:0/3449879657 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f50f809a820 con 0x7f50fc100530 2023-12-12T09:57:39.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f50e80a1510 msgr2=0x7f50e80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:39.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f50e80a1510 0x7f50e80a39d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f50ec005950 tx=0x7f50ec0058e0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc100530 msgr2=0x7f50fc1093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:39.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc100530 0x7f50fc1093e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f50f8006010 tx=0x7f50f8005e80 comp rx=0 tx=0).stop 2023-12-12T09:57:39.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 shutdown_connections 2023-12-12T09:57:39.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f50e80a1510 0x7f50e80a39d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f50fc109e60 0x7f50fc1aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.900 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50fc100530 0x7f50fc1093e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 --2- 172.21.15.42:0/3449879657 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f50fc100ef0 0x7f50fc109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:39.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 >> 172.21.15.42:0/3449879657 conn(0x7f50fc076570 msgr2=0x7f50fc108200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:39.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 shutdown_connections 2023-12-12T09:57:39.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:39.898+0000 7f5104133700 1 -- 172.21.15.42:0/3449879657 wait complete. 2023-12-12T09:57:39.940 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135906 got 244813135908 for osd.8 2023-12-12T09:57:39.940 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 -- 172.21.15.42:0/3192070428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c90102040 msgr2=0x7f6c9010e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 --2- 172.21.15.42:0/3192070428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c90102040 0x7f6c9010e430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800b0a0 tx=0x7f6c8800b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 -- 172.21.15.42:0/3192070428 shutdown_connections 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 --2- 172.21.15.42:0/3192070428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c901076f0 0x7f6c90107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 --2- 172.21.15.42:0/3192070428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c90102040 0x7f6c9010e430 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 --2- 172.21.15.42:0/3192070428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c90101680 0x7f6c90101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 -- 172.21.15.42:0/3192070428 >> 172.21.15.42:0/3192070428 conn(0x7f6c900787c0 msgr2=0x7f6c90078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 -- 172.21.15.42:0/3192070428 shutdown_connections 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 -- 172.21.15.42:0/3192070428 wait complete. 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.179+0000 7f6c9558b700 1 Processor -- start 2023-12-12T09:57:40.182 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c9558b700 1 -- start start 2023-12-12T09:57:40.184 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c9558b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c90101680 0x7f6c90197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:40.184 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c9558b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c90102040 0x7f6c90197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:40.184 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c9558b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c901076f0 0x7f6c9019c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:40.184 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c9558b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9019c910 con 0x7f6c901076f0 2023-12-12T09:57:40.184 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c9558b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9019ca80 con 0x7f6c90101680 2023-12-12T09:57:40.184 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c9558b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9019cc40 con 0x7f6c90102040 2023-12-12T09:57:40.184 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c901076f0 0x7f6c9019c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:40.184 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c901076f0 0x7f6c9019c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34342/0 (socket says 172.21.15.42:34342) 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8f7fe700 1 -- 172.21.15.42:0/275935462 learned_addr learned my addr 172.21.15.42:0/275935462 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8e7fc700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c90102040 0x7f6c90197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8effd700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c90101680 0x7f6c90197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8f7fe700 1 -- 172.21.15.42:0/275935462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c90102040 msgr2=0x7f6c90197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8f7fe700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c90102040 0x7f6c90197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8f7fe700 1 -- 172.21.15.42:0/275935462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c90101680 msgr2=0x7f6c90197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8f7fe700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c90101680 0x7f6c90197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.180+0000 7f6c8f7fe700 1 -- 172.21.15.42:0/275935462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c88009d00 con 0x7f6c901076f0 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.181+0000 7f6c8f7fe700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c901076f0 0x7f6c9019c340 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800b070 tx=0x7f6c880094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.181+0000 7f6c77fff700 1 -- 172.21.15.42:0/275935462 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800e040 con 0x7f6c901076f0 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.181+0000 7f6c77fff700 1 -- 172.21.15.42:0/275935462 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c88007990 con 0x7f6c901076f0 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.181+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9019cec0 con 0x7f6c901076f0 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.181+0000 7f6c77fff700 1 -- 172.21.15.42:0/275935462 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8801b450 con 0x7f6c901076f0 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.181+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c900739d0 con 0x7f6c901076f0 2023-12-12T09:57:40.185 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.182+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c90109b80 con 0x7f6c901076f0 2023-12-12T09:57:40.186 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.183+0000 7f6c77fff700 1 -- 172.21.15.42:0/275935462 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f6c8801d610 con 0x7f6c901076f0 2023-12-12T09:57:40.186 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.184+0000 7f6c77fff700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6c780a1590 0x7f6c780a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:40.186 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.184+0000 7f6c77fff700 1 -- 172.21.15.42:0/275935462 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6c880ced70 con 0x7f6c901076f0 2023-12-12T09:57:40.186 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.184+0000 7f6c8effd700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6c780a1590 0x7f6c780a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:40.187 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.184+0000 7f6c8effd700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6c780a1590 0x7f6c780a3a50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6c80005950 tx=0x7f6c800058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:40.189 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.187+0000 7f6c77fff700 1 -- 172.21.15.42:0/275935462 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6c88097ff0 con 0x7f6c901076f0 2023-12-12T09:57:40.214 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:40.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.395+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6c90198d30 con 0x7f6c901076f0 2023-12-12T09:57:40.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.397+0000 7f6c77fff700 1 -- 172.21.15.42:0/275935462 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c88021090 con 0x7f6c901076f0 2023-12-12T09:57:40.399 INFO:teuthology.orchestra.run.smithi042.stdout:107374182457 2023-12-12T09:57:40.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6c780a1590 msgr2=0x7f6c780a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6c780a1590 0x7f6c780a3a50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6c80005950 tx=0x7f6c800058e0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c901076f0 msgr2=0x7f6c9019c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c901076f0 0x7f6c9019c340 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800b070 tx=0x7f6c880094c0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 shutdown_connections 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6c780a1590 0x7f6c780a3a50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6c90101680 0x7f6c90197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6c901076f0 0x7f6c9019c340 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 --2- 172.21.15.42:0/275935462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c90102040 0x7f6c90197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.400+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 >> 172.21.15.42:0/275935462 conn(0x7f6c900787c0 msgr2=0x7f6c90100240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.401+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 shutdown_connections 2023-12-12T09:57:40.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:40.401+0000 7f6c9558b700 1 -- 172.21.15.42:0/275935462 wait complete. 2023-12-12T09:57:40.572 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874270 got 279172874272 for osd.9 2023-12-12T09:57:40.572 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:40 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3449879657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-12T09:57:40.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:40 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3449879657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-12T09:57:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:40 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3449879657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-12T09:57:41.222 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182455 got 107374182457 for osd.3 2023-12-12T09:57:41.222 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:41.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.344+0000 7ff13b559700 1 -- 172.21.15.42:0/2462386037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134068f70 msgr2=0x7ff134069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.344+0000 7ff13b559700 1 --2- 172.21.15.42:0/2462386037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134068f70 0x7ff134069350 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ff1300098f0 tx=0x7ff130009c00 comp rx=0 tx=0).stop 2023-12-12T09:57:41.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.344+0000 7ff13b559700 1 -- 172.21.15.42:0/2462386037 shutdown_connections 2023-12-12T09:57:41.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.344+0000 7ff13b559700 1 --2- 172.21.15.42:0/2462386037 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff13410e0e0 0x7ff1341104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.344+0000 7ff13b559700 1 --2- 172.21.15.42:0/2462386037 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff134068f70 0x7ff134069350 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.344+0000 7ff13b559700 1 --2- 172.21.15.42:0/2462386037 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134069890 0x7ff13410dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.344+0000 7ff13b559700 1 -- 172.21.15.42:0/2462386037 >> 172.21.15.42:0/2462386037 conn(0x7ff134076570 msgr2=0x7ff134076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:41.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.344+0000 7ff13b559700 1 -- 172.21.15.42:0/2462386037 shutdown_connections 2023-12-12T09:57:41.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.344+0000 7ff13b559700 1 -- 172.21.15.42:0/2462386037 wait complete. 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff13b559700 1 Processor -- start 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff13b559700 1 -- start start 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff13b559700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff134068f70 0x7ff13419f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff13b559700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134069890 0x7ff13419fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff13b559700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff13410e0e0 0x7ff134199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff13b559700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13419a140 con 0x7ff13410e0e0 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff13b559700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13419a2b0 con 0x7ff134069890 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff13b559700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13419a470 con 0x7ff134068f70 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff138af4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134069890 0x7ff13419fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff138af4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134069890 0x7ff13419fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:37426/0 (socket says 172.21.15.42:37426) 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff139af6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff13410e0e0 0x7ff134199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff138af4700 1 -- 172.21.15.42:0/1645511192 learned_addr learned my addr 172.21.15.42:0/1645511192 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.345+0000 7ff1392f5700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff134068f70 0x7ff13419f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff138af4700 1 -- 172.21.15.42:0/1645511192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff134068f70 msgr2=0x7ff13419f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff138af4700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff134068f70 0x7ff13419f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff138af4700 1 -- 172.21.15.42:0/1645511192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff13410e0e0 msgr2=0x7ff134199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff138af4700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff13410e0e0 0x7ff134199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff138af4700 1 -- 172.21.15.42:0/1645511192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff130009520 con 0x7ff134069890 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff138af4700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134069890 0x7ff13419fee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff12400e9c0 tx=0x7ff12400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff12a7fc700 1 -- 172.21.15.42:0/1645511192 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124004a30 con 0x7ff134069890 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff12a7fc700 1 -- 172.21.15.42:0/1645511192 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff12400cc80 con 0x7ff134069890 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff12a7fc700 1 -- 172.21.15.42:0/1645511192 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1240054e0 con 0x7ff134069890 2023-12-12T09:57:41.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13419a750 con 0x7ff134069890 2023-12-12T09:57:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.346+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1341a6990 con 0x7ff134069890 2023-12-12T09:57:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.347+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff134109300 con 0x7ff134069890 2023-12-12T09:57:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.348+0000 7ff12a7fc700 1 -- 172.21.15.42:0/1645511192 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff124004b90 con 0x7ff134069890 2023-12-12T09:57:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.349+0000 7ff12a7fc700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff1200a1510 0x7ff1200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.349+0000 7ff1392f5700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff1200a1510 0x7ff1200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.349+0000 7ff12a7fc700 1 -- 172.21.15.42:0/1645511192 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff124014070 con 0x7ff134069890 2023-12-12T09:57:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.349+0000 7ff1392f5700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff1200a1510 0x7ff1200a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff1300098c0 tx=0x7ff130019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:41.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.354+0000 7ff12a7fc700 1 -- 172.21.15.42:0/1645511192 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7ff124096430 con 0x7ff134069890 2023-12-12T09:57:41.372 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:41.486 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:41 smithi042 ceph-mon[98894]: pgmap v295: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 293 B/s wr, 8 op/s; 109 B/s, 1 objects/s recovering 2023-12-12T09:57:41.486 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:41 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/275935462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-12T09:57:41.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.312+0000 7f74197fc700 1 -- 172.21.15.42:0/920337745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7414100b80 msgr2=0x7f74141096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.312+0000 7f74197fc700 1 --2- 172.21.15.42:0/920337745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7414100b80 0x7f74141096c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7408009990 tx=0x7f7408009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.312+0000 7f74197fc700 1 -- 172.21.15.42:0/920337745 shutdown_connections 2023-12-12T09:57:41.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.312+0000 7f74197fc700 1 --2- 172.21.15.42:0/920337745 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74141001d0 0x7f74141005b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.312+0000 7f74197fc700 1 --2- 172.21.15.42:0/920337745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7414109da0 0x7f74141107b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.312+0000 7f74197fc700 1 --2- 172.21.15.42:0/920337745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7414100b80 0x7f74141096c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.312+0000 7f74197fc700 1 -- 172.21.15.42:0/920337745 >> 172.21.15.42:0/920337745 conn(0x7f74140757f0 msgr2=0x7f7414075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:41.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.313+0000 7f74197fc700 1 -- 172.21.15.42:0/920337745 shutdown_connections 2023-12-12T09:57:41.533 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.313+0000 7f74197fc700 1 -- 172.21.15.42:0/920337745 wait complete. 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.313+0000 7f74197fc700 1 Processor -- start 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.313+0000 7f74197fc700 1 -- start start 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f74197fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74141001d0 0x7f741419f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f74197fc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7414100b80 0x7f741419fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f74197fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7414109da0 0x7f74141999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f74197fc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741419a050 con 0x7f74141001d0 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f74197fc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741419a1c0 con 0x7f7414109da0 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f74197fc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741419a380 con 0x7f7414100b80 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f7412ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74141001d0 0x7f741419f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f7412ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74141001d0 0x7f741419f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34350/0 (socket says 172.21.15.42:34350) 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f7412ffd700 1 -- 172.21.15.42:0/1252777560 learned_addr learned my addr 172.21.15.42:0/1252777560 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f74137fe700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7414109da0 0x7f74141999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f74127fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7414100b80 0x7f741419fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.534 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f7412ffd700 1 -- 172.21.15.42:0/1252777560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7414100b80 msgr2=0x7f741419fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f7412ffd700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7414100b80 0x7f741419fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f7412ffd700 1 -- 172.21.15.42:0/1252777560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7414109da0 msgr2=0x7f74141999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f7412ffd700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7414109da0 0x7f74141999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f7412ffd700 1 -- 172.21.15.42:0/1252777560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74080095f0 con 0x7f74141001d0 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f74127fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7414100b80 0x7f741419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f7412ffd700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74141001d0 0x7f741419f8b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f740400d980 tx=0x7f740400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.314+0000 7f73fbfff700 1 -- 172.21.15.42:0/1252777560 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404004220 con 0x7f74141001d0 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.315+0000 7f73fbfff700 1 -- 172.21.15.42:0/1252777560 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7404007640 con 0x7f74141001d0 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.315+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741419a660 con 0x7f74141001d0 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.315+0000 7f73fbfff700 1 -- 172.21.15.42:0/1252777560 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404005610 con 0x7f74141001d0 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.315+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74141a68a0 con 0x7f74141001d0 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.316+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741410bf00 con 0x7f74141001d0 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.317+0000 7f73fbfff700 1 -- 172.21.15.42:0/1252777560 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7404004750 con 0x7f74141001d0 2023-12-12T09:57:41.535 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.317+0000 7f73fbfff700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73fc0a1590 0x7f73fc0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.318+0000 7f74127fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73fc0a1590 0x7f73fc0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.318+0000 7f73fbfff700 1 -- 172.21.15.42:0/1252777560 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f74040cd060 con 0x7f74141001d0 2023-12-12T09:57:41.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.318+0000 7f74127fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73fc0a1590 0x7f73fc0a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7408006010 tx=0x7f740801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:41.536 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.322+0000 7f73fbfff700 1 -- 172.21.15.42:0/1252777560 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f74041002f0 con 0x7f74141001d0 2023-12-12T09:57:41.556 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.554+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f741404f2c0 con 0x7f74141001d0 2023-12-12T09:57:41.557 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.555+0000 7f73fbfff700 1 -- 172.21.15.42:0/1252777560 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7404016020 con 0x7f74141001d0 2023-12-12T09:57:41.557 INFO:teuthology.orchestra.run.smithi042.stdout:55834574915 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73fc0a1590 msgr2=0x7f73fc0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73fc0a1590 0x7f73fc0a3a50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7408006010 tx=0x7f740801a040 comp rx=0 tx=0).stop 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74141001d0 msgr2=0x7f741419f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74141001d0 0x7f741419f8b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f740400d980 tx=0x7f740400dc90 comp rx=0 tx=0).stop 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 shutdown_connections 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73fc0a1590 0x7f73fc0a3a50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7414109da0 0x7f74141999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f74141001d0 0x7f741419f8b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 --2- 172.21.15.42:0/1252777560 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7414100b80 0x7f741419fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.559 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 >> 172.21.15.42:0/1252777560 conn(0x7f74140757f0 msgr2=0x7f74141078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 shutdown_connections 2023-12-12T09:57:41.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.557+0000 7f74197fc700 1 -- 172.21.15.42:0/1252777560 wait complete. 2023-12-12T09:57:41.605 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.601+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff13419afa0 con 0x7ff134069890 2023-12-12T09:57:41.605 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.602+0000 7ff12a7fc700 1 -- 172.21.15.42:0/1645511192 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff124099a50 con 0x7ff134069890 2023-12-12T09:57:41.605 INFO:teuthology.orchestra.run.smithi042.stdout:193273528365 2023-12-12T09:57:41.607 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff1200a1510 msgr2=0x7ff1200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.607 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff1200a1510 0x7ff1200a39d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff1300098c0 tx=0x7ff130019040 comp rx=0 tx=0).stop 2023-12-12T09:57:41.607 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134069890 msgr2=0x7ff13419fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.607 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134069890 0x7ff13419fee0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff12400e9c0 tx=0x7ff12400ecd0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 shutdown_connections 2023-12-12T09:57:41.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff1200a1510 0x7ff1200a39d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff13410e0e0 0x7ff134199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff134068f70 0x7ff13419f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 --2- 172.21.15.42:0/1645511192 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134069890 0x7ff13419fee0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 >> 172.21.15.42:0/1645511192 conn(0x7ff134076570 msgr2=0x7ff13410f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:41.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 shutdown_connections 2023-12-12T09:57:41.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.604+0000 7ff13b559700 1 -- 172.21.15.42:0/1645511192 wait complete. 2023-12-12T09:57:41.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.675+0000 7f1104c9c700 1 -- 172.21.15.42:0/3504190764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100101680 msgr2=0x7f1100101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.675+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3504190764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100101680 0x7f1100101b00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009990 tx=0x7f10e8009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.675+0000 7f1104c9c700 1 -- 172.21.15.42:0/3504190764 shutdown_connections 2023-12-12T09:57:41.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.675+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3504190764 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1100102040 0x7f110010e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.675+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3504190764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1100107660 0x7f1100107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.675+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3504190764 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100101680 0x7f1100101b00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.675+0000 7f1104c9c700 1 -- 172.21.15.42:0/3504190764 >> 172.21.15.42:0/3504190764 conn(0x7f11000786f0 msgr2=0x7f1100078b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:41.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.675+0000 7f1104c9c700 1 -- 172.21.15.42:0/3504190764 shutdown_connections 2023-12-12T09:57:41.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.675+0000 7f1104c9c700 1 -- 172.21.15.42:0/3504190764 wait complete. 2023-12-12T09:57:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.676+0000 7f1104c9c700 1 Processor -- start 2023-12-12T09:57:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.676+0000 7f1104c9c700 1 -- start start 2023-12-12T09:57:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.676+0000 7f1104c9c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1100101680 0x7f1100197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.676+0000 7f1104c9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1100102040 0x7f1100197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.676+0000 7f1104c9c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100107660 0x7f110019c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.676+0000 7f1104c9c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110019c940 con 0x7f1100107660 2023-12-12T09:57:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.676+0000 7f1104c9c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110019cab0 con 0x7f1100102040 2023-12-12T09:57:41.681 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.676+0000 7f1104c9c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110019cc70 con 0x7f1100101680 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fdd9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1100102040 0x7f1100197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100107660 0x7f110019c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fdd9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1100102040 0x7f1100197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:37440/0 (socket says 172.21.15.42:37440) 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fdd9b700 1 -- 172.21.15.42:0/3953687125 learned_addr learned my addr 172.21.15.42:0/3953687125 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100107660 0x7f110019c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34382/0 (socket says 172.21.15.42:34382) 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fe59c700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1100101680 0x7f1100197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fed9d700 1 -- 172.21.15.42:0/3953687125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1100101680 msgr2=0x7f1100197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fed9d700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1100101680 0x7f1100197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fed9d700 1 -- 172.21.15.42:0/3953687125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1100102040 msgr2=0x7f1100197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fed9d700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1100102040 0x7f1100197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fed9d700 1 -- 172.21.15.42:0/3953687125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e80095f0 con 0x7f1100107660 2023-12-12T09:57:41.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fdd9b700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1100102040 0x7f1100197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:41.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10fed9d700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100107660 0x7f110019c370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f10f800efc0 tx=0x7f10f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:41.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10f77fe700 1 -- 172.21.15.42:0/3953687125 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8018070 con 0x7f1100107660 2023-12-12T09:57:41.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110019cef0 con 0x7f1100107660 2023-12-12T09:57:41.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10f77fe700 1 -- 172.21.15.42:0/3953687125 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f10f800f040 con 0x7f1100107660 2023-12-12T09:57:41.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.677+0000 7f10f77fe700 1 -- 172.21.15.42:0/3953687125 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f800cd40 con 0x7f1100107660 2023-12-12T09:57:41.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.678+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110019d1d0 con 0x7f1100107660 2023-12-12T09:57:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.678+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1100109b10 con 0x7f1100107660 2023-12-12T09:57:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.680+0000 7f10f77fe700 1 -- 172.21.15.42:0/3953687125 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f10f8003a00 con 0x7f1100107660 2023-12-12T09:57:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.680+0000 7f10f77fe700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f10ec0a1510 0x7f10ec0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.681+0000 7f10fe59c700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f10ec0a1510 0x7f10ec0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.681+0000 7f10f77fe700 1 -- 172.21.15.42:0/3953687125 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f10f80cd700 con 0x7f1100107660 2023-12-12T09:57:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.681+0000 7f10fe59c700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f10ec0a1510 0x7f10ec0a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005950 tx=0x7f10f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:41.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.685+0000 7f10f77fe700 1 -- 172.21.15.42:0/3953687125 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f10f8096980 con 0x7f1100107660 2023-12-12T09:57:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:41 smithi111 ceph-mon[102565]: pgmap v295: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 293 B/s wr, 8 op/s; 109 B/s, 1 objects/s recovering 2023-12-12T09:57:41.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:41 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/275935462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-12T09:57:41.783 INFO:tasks.cephadm.ceph_manager.ceph:need seq 214748364838 got 214748364840 for osd.7 2023-12-12T09:57:41.784 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:41 smithi173 ceph-mon[103308]: pgmap v295: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 293 B/s wr, 8 op/s; 109 B/s, 1 objects/s recovering 2023-12-12T09:57:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:41 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/275935462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-12T09:57:41.908 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.904+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1100102d00 con 0x7f1100107660 2023-12-12T09:57:41.908 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.905+0000 7f10f77fe700 1 -- 172.21.15.42:0/3953687125 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10f8099fa0 con 0x7f1100107660 2023-12-12T09:57:41.908 INFO:teuthology.orchestra.run.smithi042.stdout:158913790002 2023-12-12T09:57:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f10ec0a1510 msgr2=0x7f10ec0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f10ec0a1510 0x7f10ec0a39d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10f0005950 tx=0x7f10f000b410 comp rx=0 tx=0).stop 2023-12-12T09:57:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100107660 msgr2=0x7f110019c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100107660 0x7f110019c370 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f10f800efc0 tx=0x7f10f800c3a0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.911 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 shutdown_connections 2023-12-12T09:57:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f10ec0a1510 0x7f10ec0a39d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1100107660 0x7f110019c370 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1100101680 0x7f1100197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 --2- 172.21.15.42:0/3953687125 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1100102040 0x7f1100197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 >> 172.21.15.42:0/3953687125 conn(0x7f11000786f0 msgr2=0x7f1100100530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 shutdown_connections 2023-12-12T09:57:41.912 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:41.908+0000 7f1104c9c700 1 -- 172.21.15.42:0/3953687125 wait complete. 2023-12-12T09:57:42.236 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574913 got 55834574915 for osd.1 2023-12-12T09:57:42.236 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:42.505 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528363 got 193273528365 for osd.6 2023-12-12T09:57:42.505 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:42.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:42 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1252777560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-12T09:57:42.655 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:42 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1645511192' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-12T09:57:42.655 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:42 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3953687125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-12T09:57:42.655 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:42 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.655 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:42 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.655 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:42 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.655 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:42 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:42 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1252777560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-12T09:57:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:42 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1645511192' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-12T09:57:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:42 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3953687125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-12T09:57:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:42 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:42 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:42 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:42 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:42 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1252777560' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-12T09:57:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:42 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1645511192' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-12T09:57:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:42 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3953687125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-12T09:57:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:42 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:42 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:42 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:42 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:43.072 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790000 got 158913790002 for osd.5 2023-12-12T09:57:43.072 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:43.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.167+0000 7fd88b14c700 1 -- 172.21.15.42:0/3089235853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884068f70 msgr2=0x7fd8840693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:43.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.167+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3089235853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884068f70 0x7fd8840693f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd8700099c0 tx=0x7fd870009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 -- 172.21.15.42:0/3089235853 shutdown_connections 2023-12-12T09:57:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3089235853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd884107620 0x7fd884107a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3089235853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd884069930 0x7fd88410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3089235853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884068f70 0x7fd8840693f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 -- 172.21.15.42:0/3089235853 >> 172.21.15.42:0/3089235853 conn(0x7fd884076570 msgr2=0x7fd884076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 -- 172.21.15.42:0/3089235853 shutdown_connections 2023-12-12T09:57:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 -- 172.21.15.42:0/3089235853 wait complete. 2023-12-12T09:57:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 Processor -- start 2023-12-12T09:57:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 -- start start 2023-12-12T09:57:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd884068f70 0x7fd88419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884069930 0x7fd88419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd884107620 0x7fd884197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.168+0000 7fd88b14c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884197ed0 con 0x7fd884069930 2023-12-12T09:57:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd88b14c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884198040 con 0x7fd884068f70 2023-12-12T09:57:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd88b14c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd884198200 con 0x7fd884107620 2023-12-12T09:57:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd883fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884069930 0x7fd88419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd883fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884069930 0x7fd88419dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34392/0 (socket says 172.21.15.42:34392) 2023-12-12T09:57:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd883fff700 1 -- 172.21.15.42:0/3240298075 learned_addr learned my addr 172.21.15.42:0/3240298075 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd8896e9700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd884107620 0x7fd884197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd888ee8700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd884068f70 0x7fd88419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd883fff700 1 -- 172.21.15.42:0/3240298075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd884107620 msgr2=0x7fd884197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd883fff700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd884107620 0x7fd884197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd883fff700 1 -- 172.21.15.42:0/3240298075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd884068f70 msgr2=0x7fd88419d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd883fff700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd884068f70 0x7fd88419d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd883fff700 1 -- 172.21.15.42:0/3240298075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8700095f0 con 0x7fd884069930 2023-12-12T09:57:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd883fff700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884069930 0x7fd88419dc10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd87000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd881ffb700 1 -- 172.21.15.42:0/3240298075 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87001d070 con 0x7fd884069930 2023-12-12T09:57:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd884198480 con 0x7fd884069930 2023-12-12T09:57:43.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd881ffb700 1 -- 172.21.15.42:0/3240298075 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd870022470 con 0x7fd884069930 2023-12-12T09:57:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.169+0000 7fd881ffb700 1 -- 172.21.15.42:0/3240298075 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870017650 con 0x7fd884069930 2023-12-12T09:57:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.170+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8841a46c0 con 0x7fd884069930 2023-12-12T09:57:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.170+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd884109a00 con 0x7fd884069930 2023-12-12T09:57:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.172+0000 7fd881ffb700 1 -- 172.21.15.42:0/3240298075 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fd870022850 con 0x7fd884069930 2023-12-12T09:57:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.173+0000 7fd881ffb700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd8740a1510 0x7fd8740a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.173+0000 7fd888ee8700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd8740a1510 0x7fd8740a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.173+0000 7fd881ffb700 1 -- 172.21.15.42:0/3240298075 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd8700cdf90 con 0x7fd884069930 2023-12-12T09:57:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.173+0000 7fd888ee8700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd8740a1510 0x7fd8740a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd878009be0 tx=0x7fd878009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.176+0000 7fd881ffb700 1 -- 172.21.15.42:0/3240298075 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd870097320 con 0x7fd884069930 2023-12-12T09:57:43.410 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.406+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd884198cd0 con 0x7fd884069930 2023-12-12T09:57:43.410 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.407+0000 7fd881ffb700 1 -- 172.21.15.42:0/3240298075 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd87009a940 con 0x7fd884069930 2023-12-12T09:57:43.410 INFO:teuthology.orchestra.run.smithi042.stdout:34359738440 2023-12-12T09:57:43.412 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd8740a1510 msgr2=0x7fd8740a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:43.412 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd8740a1510 0x7fd8740a39d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd878009be0 tx=0x7fd878009260 comp rx=0 tx=0).stop 2023-12-12T09:57:43.412 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884069930 msgr2=0x7fd88419dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:43.412 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884069930 0x7fd88419dc10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd870000c00 tx=0x7fd87000f690 comp rx=0 tx=0).stop 2023-12-12T09:57:43.412 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 shutdown_connections 2023-12-12T09:57:43.412 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fd8740a1510 0x7fd8740a39d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd884068f70 0x7fd88419d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd884107620 0x7fd884197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 --2- 172.21.15.42:0/3240298075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd884069930 0x7fd88419dc10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 >> 172.21.15.42:0/3240298075 conn(0x7fd884076570 msgr2=0x7fd8840ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 shutdown_connections 2023-12-12T09:57:43.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:43.409+0000 7fd88b14c700 1 -- 172.21.15.42:0/3240298075 wait complete. 2023-12-12T09:57:43.559 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:43 smithi042 ceph-mon[98894]: pgmap v296: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s; 95 B/s, 1 objects/s recovering 2023-12-12T09:57:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:43 smithi111 ceph-mon[102565]: pgmap v296: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s; 95 B/s, 1 objects/s recovering 2023-12-12T09:57:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:43 smithi173 ceph-mon[103308]: pgmap v296: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 255 B/s wr, 9 op/s; 95 B/s, 1 objects/s recovering 2023-12-12T09:57:43.825 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738438 got 34359738440 for osd.0 2023-12-12T09:57:43.825 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.581+0000 7f7685cff700 1 -- 172.21.15.42:0/2265518544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680104160 msgr2=0x7f7680110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.581+0000 7f7685cff700 1 --2- 172.21.15.42:0/2265518544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680104160 0x7f7680110670 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f767800b0a0 tx=0x7f767800b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.581+0000 7f7685cff700 1 -- 172.21.15.42:0/2265518544 shutdown_connections 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.581+0000 7f7685cff700 1 --2- 172.21.15.42:0/2265518544 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76801097a0 0x7f7680109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.581+0000 7f7685cff700 1 --2- 172.21.15.42:0/2265518544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7680104160 0x7f7680110670 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.581+0000 7f7685cff700 1 --2- 172.21.15.42:0/2265518544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76801037a0 0x7f7680103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.581+0000 7f7685cff700 1 -- 172.21.15.42:0/2265518544 >> 172.21.15.42:0/2265518544 conn(0x7f76800ff290 msgr2=0x7f76801016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.581+0000 7f7685cff700 1 -- 172.21.15.42:0/2265518544 shutdown_connections 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.581+0000 7f7685cff700 1 -- 172.21.15.42:0/2265518544 wait complete. 2023-12-12T09:57:44.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f7685cff700 1 Processor -- start 2023-12-12T09:57:44.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f7685cff700 1 -- start start 2023-12-12T09:57:44.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f7685cff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76801037a0 0x7f768019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:44.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f7685cff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7680104160 0x7f768019fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:44.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f7685cff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76801097a0 0x7f7680199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:44.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f7685cff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768019a110 con 0x7f76801097a0 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f7685cff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768019a280 con 0x7f76801037a0 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f7685cff700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768019a440 con 0x7f7680104160 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f767ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76801097a0 0x7f7680199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f767ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76801097a0 0x7f7680199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34408/0 (socket says 172.21.15.42:34408) 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f767ffff700 1 -- 172.21.15.42:0/3960626929 learned_addr learned my addr 172.21.15.42:0/3960626929 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f767f7fe700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76801037a0 0x7f768019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.582+0000 7f767effd700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7680104160 0x7f768019fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767ffff700 1 -- 172.21.15.42:0/3960626929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7680104160 msgr2=0x7f768019fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767ffff700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7680104160 0x7f768019fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767ffff700 1 -- 172.21.15.42:0/3960626929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76801037a0 msgr2=0x7f768019f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767ffff700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76801037a0 0x7f768019f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767ffff700 1 -- 172.21.15.42:0/3960626929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7678009d00 con 0x7f76801097a0 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767f7fe700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76801037a0 0x7f768019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:44.587 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767effd700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7680104160 0x7f768019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767ffff700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76801097a0 0x7f7680199ab0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f767800b070 tx=0x7f7678007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767cff9700 1 -- 172.21.15.42:0/3960626929 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767800e040 con 0x7f76801097a0 2023-12-12T09:57:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767cff9700 1 -- 172.21.15.42:0/3960626929 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7678009440 con 0x7f76801097a0 2023-12-12T09:57:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768019a6c0 con 0x7f76801097a0 2023-12-12T09:57:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f767cff9700 1 -- 172.21.15.42:0/3960626929 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767801b6b0 con 0x7f76801097a0 2023-12-12T09:57:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.583+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76801a6910 con 0x7f76801097a0 2023-12-12T09:57:44.588 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.584+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768010bdc0 con 0x7f76801097a0 2023-12-12T09:57:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.586+0000 7f767cff9700 1 -- 172.21.15.42:0/3960626929 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f767801d480 con 0x7f76801097a0 2023-12-12T09:57:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.586+0000 7f767cff9700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f766c0a1510 0x7f766c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.586+0000 7f767cff9700 1 -- 172.21.15.42:0/3960626929 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f76780cee10 con 0x7f76801097a0 2023-12-12T09:57:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.586+0000 7f767f7fe700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f766c0a1510 0x7f766c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:44.589 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.587+0000 7f767f7fe700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f766c0a1510 0x7f766c0a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7670005fd0 tx=0x7f767000a5e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:44.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.589+0000 7f767cff9700 1 -- 172.21.15.42:0/3960626929 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7678098090 con 0x7f76801097a0 2023-12-12T09:57:44.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:44 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3240298075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-12T09:57:44.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:44 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3240298075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-12T09:57:44.800 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.798+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f7680104ee0 con 0x7f76801097a0 2023-12-12T09:57:44.801 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.798+0000 7f767cff9700 1 -- 172.21.15.42:0/3960626929 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f767801de60 con 0x7f76801097a0 2023-12-12T09:57:44.801 INFO:teuthology.orchestra.run.smithi042.stdout:300647710748 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f766c0a1510 msgr2=0x7f766c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f766c0a1510 0x7f766c0a39d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7670005fd0 tx=0x7f767000a5e0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76801097a0 msgr2=0x7f7680199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76801097a0 0x7f7680199ab0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f767800b070 tx=0x7f7678007920 comp rx=0 tx=0).stop 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 shutdown_connections 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f766c0a1510 0x7f766c0a39d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76801037a0 0x7f768019f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76801097a0 0x7f7680199ab0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 --2- 172.21.15.42:0/3960626929 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7680104160 0x7f768019fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.803 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 >> 172.21.15.42:0/3960626929 conn(0x7f76800ff290 msgr2=0x7f76800ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:44.804 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 shutdown_connections 2023-12-12T09:57:44.804 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.801+0000 7f7685cff700 1 -- 172.21.15.42:0/3960626929 wait complete. 2023-12-12T09:57:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:44 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3240298075' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-12T09:57:44.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.872+0000 7f0774102700 1 -- 172.21.15.42:0/125010454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c068f70 msgr2=0x7f076c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:44.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.872+0000 7f0774102700 1 --2- 172.21.15.42:0/125010454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c068f70 0x7f076c069350 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f076400b0a0 tx=0x7f076400b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.872+0000 7f0774102700 1 -- 172.21.15.42:0/125010454 shutdown_connections 2023-12-12T09:57:44.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.872+0000 7f0774102700 1 --2- 172.21.15.42:0/125010454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f076c10e0e0 0x7f076c1104d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.872+0000 7f0774102700 1 --2- 172.21.15.42:0/125010454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c068f70 0x7f076c069350 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.872+0000 7f0774102700 1 --2- 172.21.15.42:0/125010454 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f076c069890 0x7f076c10dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.872+0000 7f0774102700 1 -- 172.21.15.42:0/125010454 >> 172.21.15.42:0/125010454 conn(0x7f076c076570 msgr2=0x7f076c076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:44.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.872+0000 7f0774102700 1 -- 172.21.15.42:0/125010454 shutdown_connections 2023-12-12T09:57:44.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.872+0000 7f0774102700 1 -- 172.21.15.42:0/125010454 wait complete. 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.873+0000 7f0774102700 1 Processor -- start 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.873+0000 7f0774102700 1 -- start start 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f0774102700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f076c068f70 0x7f076c19fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f0774102700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c069890 0x7f076c19ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f0774102700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f076c10e0e0 0x7f076c199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f0774102700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c19a1f0 con 0x7f076c069890 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f0774102700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c19a360 con 0x7f076c10e0e0 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f0774102700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c19a520 con 0x7f076c068f70 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077169d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c069890 0x7f076c19ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077169d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c069890 0x7f076c19ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34420/0 (socket says 172.21.15.42:34420) 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077169d700 1 -- 172.21.15.42:0/3071580061 learned_addr learned my addr 172.21.15.42:0/3071580061 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077269f700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f076c10e0e0 0x7f076c199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:44.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077169d700 1 -- 172.21.15.42:0/3071580061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f076c068f70 msgr2=0x7f076c19fa50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-12T09:57:44.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077169d700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f076c068f70 0x7f076c19fa50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077169d700 1 -- 172.21.15.42:0/3071580061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f076c10e0e0 msgr2=0x7f076c199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:44.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077169d700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f076c10e0e0 0x7f076c199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:44.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077169d700 1 -- 172.21.15.42:0/3071580061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0764009d00 con 0x7f076c069890 2023-12-12T09:57:44.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.874+0000 7f077269f700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f076c10e0e0 0x7f076c199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:44.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.875+0000 7f077169d700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c069890 0x7f076c19ff90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f075c00d980 tx=0x7f075c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:44.879 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.877+0000 7f0762ffd700 1 -- 172.21.15.42:0/3071580061 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c004220 con 0x7f076c069890 2023-12-12T09:57:44.879 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.877+0000 7f0762ffd700 1 -- 172.21.15.42:0/3071580061 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f075c004cf0 con 0x7f076c069890 2023-12-12T09:57:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.877+0000 7f0762ffd700 1 -- 172.21.15.42:0/3071580061 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c005720 con 0x7f076c069890 2023-12-12T09:57:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.877+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076c19a800 con 0x7f076c069890 2023-12-12T09:57:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.877+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c1a6a40 con 0x7f076c069890 2023-12-12T09:57:44.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.878+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076c109300 con 0x7f076c069890 2023-12-12T09:57:44.882 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.880+0000 7f0762ffd700 1 -- 172.21.15.42:0/3071580061 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f075c004410 con 0x7f076c069890 2023-12-12T09:57:44.883 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.881+0000 7f0762ffd700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f07580a1660 0x7f07580a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:44.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.881+0000 7f0771e9e700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f07580a1660 0x7f07580a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:44.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.881+0000 7f0762ffd700 1 -- 172.21.15.42:0/3071580061 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f075c0cd2f0 con 0x7f076c069890 2023-12-12T09:57:44.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.882+0000 7f0771e9e700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f07580a1660 0x7f07580a3b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f076400b070 tx=0x7f07640060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:44.886 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:44.884+0000 7f0762ffd700 1 -- 172.21.15.42:0/3071580061 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f075c096600 con 0x7f076c069890 2023-12-12T09:57:45.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.089+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f076c19afe0 con 0x7f076c069890 2023-12-12T09:57:45.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.089+0000 7f0762ffd700 1 -- 172.21.15.42:0/3071580061 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f075c016070 con 0x7f076c069890 2023-12-12T09:57:45.091 INFO:teuthology.orchestra.run.smithi042.stdout:335007449112 2023-12-12T09:57:45.093 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f07580a1660 msgr2=0x7f07580a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:45.093 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f07580a1660 0x7f07580a3b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f076400b070 tx=0x7f07640060c0 comp rx=0 tx=0).stop 2023-12-12T09:57:45.093 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c069890 msgr2=0x7f076c19ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:45.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c069890 0x7f076c19ff90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f075c00d980 tx=0x7f075c00dc90 comp rx=0 tx=0).stop 2023-12-12T09:57:45.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 shutdown_connections 2023-12-12T09:57:45.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f07580a1660 0x7f07580a3b20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:45.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f076c10e0e0 0x7f076c199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:45.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f076c068f70 0x7f076c19fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:45.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 --2- 172.21.15.42:0/3071580061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f076c069890 0x7f076c19ff90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:45.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 >> 172.21.15.42:0/3071580061 conn(0x7f076c076570 msgr2=0x7f076c10f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:45.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 shutdown_connections 2023-12-12T09:57:45.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:45.091+0000 7f0774102700 1 -- 172.21.15.42:0/3071580061 wait complete. 2023-12-12T09:57:45.218 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710746 got 300647710748 for osd.10 2023-12-12T09:57:45.219 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:45.571 INFO:tasks.cephadm.ceph_manager.ceph:need seq 335007449110 got 335007449112 for osd.11 2023-12-12T09:57:45.571 DEBUG:teuthology.parallel:result is None 2023-12-12T09:57:45.571 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-12-12T09:57:45.571 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph pg dump --format=json 2023-12-12T09:57:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:45 smithi111 ceph-mon[102565]: pgmap v297: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2023-12-12T09:57:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:45 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3960626929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-12T09:57:45.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:45 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3071580061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-12T09:57:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:45 smithi173 ceph-mon[103308]: pgmap v297: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2023-12-12T09:57:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:45 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3960626929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-12T09:57:45.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:45 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3071580061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-12T09:57:45.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:45 smithi042 ceph-mon[98894]: pgmap v297: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s; 0 B/s, 0 objects/s recovering 2023-12-12T09:57:45.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:45 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3960626929' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-12T09:57:45.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:45 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3071580061' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-12T09:57:46.184 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:47.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.588+0000 7fc83a6a5700 1 -- 172.21.15.42:0/3517767952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 msgr2=0x7fc834110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:47.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.588+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/3517767952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 0x7fc834110670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00b0a0 tx=0x7fc82c00b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.588+0000 7fc83a6a5700 1 -- 172.21.15.42:0/3517767952 shutdown_connections 2023-12-12T09:57:47.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.588+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/3517767952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8341097a0 0x7fc834109b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.588+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/3517767952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 0x7fc834110670 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.588+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/3517767952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8341037a0 0x7fc834103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.588+0000 7fc83a6a5700 1 -- 172.21.15.42:0/3517767952 >> 172.21.15.42:0/3517767952 conn(0x7fc8340ff250 msgr2=0x7fc834101670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.588+0000 7fc83a6a5700 1 -- 172.21.15.42:0/3517767952 shutdown_connections 2023-12-12T09:57:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.588+0000 7fc83a6a5700 1 -- 172.21.15.42:0/3517767952 wait complete. 2023-12-12T09:57:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc83a6a5700 1 Processor -- start 2023-12-12T09:57:47.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc83a6a5700 1 -- start start 2023-12-12T09:57:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc83a6a5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8341037a0 0x7fc834199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc83a6a5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 0x7fc83419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc83a6a5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8341097a0 0x7fc83419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc83a6a5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419db40 con 0x7fc834104160 2023-12-12T09:57:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc83a6a5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419dcb0 con 0x7fc8341037a0 2023-12-12T09:57:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc83a6a5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc83419de70 con 0x7fc8341097a0 2023-12-12T09:57:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc8337fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 0x7fc83419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc8337fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 0x7fc83419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34434/0 (socket says 172.21.15.42:34434) 2023-12-12T09:57:47.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc833fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8341037a0 0x7fc834199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc8337fe700 1 -- 172.21.15.42:0/1575135467 learned_addr learned my addr 172.21.15.42:0/1575135467 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.589+0000 7fc838c42700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8341097a0 0x7fc83419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc8337fe700 1 -- 172.21.15.42:0/1575135467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8341097a0 msgr2=0x7fc83419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc8337fe700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8341097a0 0x7fc83419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc8337fe700 1 -- 172.21.15.42:0/1575135467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8341037a0 msgr2=0x7fc834199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc8337fe700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8341037a0 0x7fc834199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc8337fe700 1 -- 172.21.15.42:0/1575135467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82c009d00 con 0x7fc834104160 2023-12-12T09:57:47.596 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc8337fe700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 0x7fc83419a240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d950 tx=0x7fc81c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc8317fa700 1 -- 172.21.15.42:0/1575135467 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00f890 con 0x7fc834104160 2023-12-12T09:57:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc83419e150 con 0x7fc834104160 2023-12-12T09:57:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc8317fa700 1 -- 172.21.15.42:0/1575135467 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc81c010460 con 0x7fc834104160 2023-12-12T09:57:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc8317fa700 1 -- 172.21.15.42:0/1575135467 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81c00e610 con 0x7fc834104160 2023-12-12T09:57:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.590+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8341a6aa0 con 0x7fc834104160 2023-12-12T09:57:47.597 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.591+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83410bd30 con 0x7fc834104160 2023-12-12T09:57:47.599 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.592+0000 7fc8317fa700 1 -- 172.21.15.42:0/1575135467 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fc81c0108d0 con 0x7fc834104160 2023-12-12T09:57:47.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.593+0000 7fc8317fa700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8200a1510 0x7fc8200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:47.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.593+0000 7fc833fff700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8200a1510 0x7fc8200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:47.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.593+0000 7fc8317fa700 1 -- 172.21.15.42:0/1575135467 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc81c0ccf90 con 0x7fc834104160 2023-12-12T09:57:47.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.593+0000 7fc833fff700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8200a1510 0x7fc8200a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc824005950 tx=0x7fc82400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:47.601 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.597+0000 7fc8317fa700 1 -- 172.21.15.42:0/1575135467 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fc81c096120 con 0x7fc834104160 2023-12-12T09:57:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:47 smithi111 ceph-mon[102565]: pgmap v298: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-12T09:57:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:47.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:57:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:57:47.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:47 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.790+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc83419afd0 con 0x7fc8200a1510 2023-12-12T09:57:47.808 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.806+0000 7fc8317fa700 1 -- 172.21.15.42:0/1575135467 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426894 (secure 0 0 0) 0x7fc83419afd0 con 0x7fc8200a1510 2023-12-12T09:57:47.813 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:47.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8200a1510 msgr2=0x7fc8200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:47.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8200a1510 0x7fc8200a39d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc824005950 tx=0x7fc82400b410 comp rx=0 tx=0).stop 2023-12-12T09:57:47.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 msgr2=0x7fc83419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 0x7fc83419a240 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc81c00d950 tx=0x7fc81c00dc60 comp rx=0 tx=0).stop 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 shutdown_connections 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc8200a1510 0x7fc8200a39d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc8341037a0 0x7fc834199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8341097a0 0x7fc83419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 --2- 172.21.15.42:0/1575135467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc834104160 0x7fc83419a240 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 >> 172.21.15.42:0/1575135467 conn(0x7fc8340ff250 msgr2=0x7fc8340ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 shutdown_connections 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:47.809+0000 7fc83a6a5700 1 -- 172.21.15.42:0/1575135467 wait complete. 2023-12-12T09:57:47.818 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2023-12-12T09:57:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:47 smithi173 ceph-mon[103308]: pgmap v298: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-12T09:57:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:47.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:57:47.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:57:47.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:47 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:47.838 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:47 smithi042 ceph-mon[98894]: pgmap v298: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2023-12-12T09:57:47.838 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:47.838 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:47.838 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:57:47.839 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:57:47.839 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:47 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' 2023-12-12T09:57:48.248 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2023-12-12T09:57:45.825650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657664,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":535,"num_read_kb":395,"num_write":422,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181880,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338168,"kb_used_data":13920,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603656,"statfs":{"total":1151940427776,"available":1151594143744,"internally_reserved":0,"allocated":14254080,"data_stored":7833973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":66,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003454"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366763+0000","last_change":"2023-12-12T09:56:38.273952+0000","last_active":"2023-12-12T09:57:28.366763+0000","last_peered":"2023-12-12T09:57:28.366763+0000","last_clean":"2023-12-12T09:57:28.366763+0000","last_became_active":"2023-12-12T09:56:38.273609+0000","last_became_peered":"2023-12-12T09:56:38.273609+0000","last_unstale":"2023-12-12T09:57:28.366763+0000","last_undegraded":"2023-12-12T09:57:28.366763+0000","last_fullsized":"2023-12-12T09:57:28.366763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:23:43.144987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371202+0000","last_change":"2023-12-12T09:56:38.278719+0000","last_active":"2023-12-12T09:57:28.371202+0000","last_peered":"2023-12-12T09:57:28.371202+0000","last_clean":"2023-12-12T09:57:28.371202+0000","last_became_active":"2023-12-12T09:56:38.278404+0000","last_became_peered":"2023-12-12T09:56:38.278404+0000","last_unstale":"2023-12-12T09:57:28.371202+0000","last_undegraded":"2023-12-12T09:57:28.371202+0000","last_fullsized":"2023-12-12T09:57:28.371202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:34:01.139106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370876+0000","last_change":"2023-12-12T09:56:38.274276+0000","last_active":"2023-12-12T09:57:28.370876+0000","last_peered":"2023-12-12T09:57:28.370876+0000","last_clean":"2023-12-12T09:57:28.370876+0000","last_became_active":"2023-12-12T09:56:38.273974+0000","last_became_peered":"2023-12-12T09:56:38.273974+0000","last_unstale":"2023-12-12T09:57:28.370876+0000","last_undegraded":"2023-12-12T09:57:28.370876+0000","last_fullsized":"2023-12-12T09:57:28.370876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:40:55.979238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.368005+0000","last_change":"2023-12-12T09:56:38.276021+0000","last_active":"2023-12-12T09:57:28.368005+0000","last_peered":"2023-12-12T09:57:28.368005+0000","last_clean":"2023-12-12T09:57:28.368005+0000","last_became_active":"2023-12-12T09:56:38.275305+0000","last_became_peered":"2023-12-12T09:56:38.275305+0000","last_unstale":"2023-12-12T09:57:28.368005+0000","last_undegraded":"2023-12-12T09:57:28.368005+0000","last_fullsized":"2023-12-12T09:57:28.368005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:32:28.839430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367304+0000","last_change":"2023-12-12T09:56:38.286720+0000","last_active":"2023-12-12T09:57:28.367304+0000","last_peered":"2023-12-12T09:57:28.367304+0000","last_clean":"2023-12-12T09:57:28.367304+0000","last_became_active":"2023-12-12T09:56:38.286410+0000","last_became_peered":"2023-12-12T09:56:38.286410+0000","last_unstale":"2023-12-12T09:57:28.367304+0000","last_undegraded":"2023-12-12T09:57:28.367304+0000","last_fullsized":"2023-12-12T09:57:28.367304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:07:42.131132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365219+0000","last_change":"2023-12-12T09:56:38.273350+0000","last_active":"2023-12-12T09:57:28.365219+0000","last_peered":"2023-12-12T09:57:28.365219+0000","last_clean":"2023-12-12T09:57:28.365219+0000","last_became_active":"2023-12-12T09:56:38.273077+0000","last_became_peered":"2023-12-12T09:56:38.273077+0000","last_unstale":"2023-12-12T09:57:28.365219+0000","last_undegraded":"2023-12-12T09:57:28.365219+0000","last_fullsized":"2023-12-12T09:57:28.365219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:28.081132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370577+0000","last_change":"2023-12-12T09:56:38.278212+0000","last_active":"2023-12-12T09:57:28.370577+0000","last_peered":"2023-12-12T09:57:28.370577+0000","last_clean":"2023-12-12T09:57:28.370577+0000","last_became_active":"2023-12-12T09:56:38.277913+0000","last_became_peered":"2023-12-12T09:56:38.277913+0000","last_unstale":"2023-12-12T09:57:28.370577+0000","last_undegraded":"2023-12-12T09:57:28.370577+0000","last_fullsized":"2023-12-12T09:57:28.370577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:39:11.893381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369879+0000","last_change":"2023-12-12T09:56:38.286625+0000","last_active":"2023-12-12T09:57:28.369879+0000","last_peered":"2023-12-12T09:57:28.369879+0000","last_clean":"2023-12-12T09:57:28.369879+0000","last_became_active":"2023-12-12T09:56:38.286035+0000","last_became_peered":"2023-12-12T09:56:38.286035+0000","last_unstale":"2023-12-12T09:57:28.369879+0000","last_undegraded":"2023-12-12T09:57:28.369879+0000","last_fullsized":"2023-12-12T09:57:28.369879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:13:03.723529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372066+0000","last_change":"2023-12-12T09:56:38.281533+0000","last_active":"2023-12-12T09:57:28.372066+0000","last_peered":"2023-12-12T09:57:28.372066+0000","last_clean":"2023-12-12T09:57:28.372066+0000","last_became_active":"2023-12-12T09:56:38.281228+0000","last_became_peered":"2023-12-12T09:56:38.281228+0000","last_unstale":"2023-12-12T09:57:28.372066+0000","last_undegraded":"2023-12-12T09:57:28.372066+0000","last_fullsized":"2023-12-12T09:57:28.372066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:49:34.075982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365595+0000","last_change":"2023-12-12T09:56:38.274599+0000","last_active":"2023-12-12T09:57:28.365595+0000","last_peered":"2023-12-12T09:57:28.365595+0000","last_clean":"2023-12-12T09:57:28.365595+0000","last_became_active":"2023-12-12T09:56:38.274309+0000","last_became_peered":"2023-12-12T09:56:38.274309+0000","last_unstale":"2023-12-12T09:57:28.365595+0000","last_undegraded":"2023-12-12T09:57:28.365595+0000","last_fullsized":"2023-12-12T09:57:28.365595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:05:06.054345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370697+0000","last_change":"2023-12-12T09:56:38.279348+0000","last_active":"2023-12-12T09:57:28.370697+0000","last_peered":"2023-12-12T09:57:28.370697+0000","last_clean":"2023-12-12T09:57:28.370697+0000","last_became_active":"2023-12-12T09:56:38.278843+0000","last_became_peered":"2023-12-12T09:56:38.278843+0000","last_unstale":"2023-12-12T09:57:28.370697+0000","last_undegraded":"2023-12-12T09:57:28.370697+0000","last_fullsized":"2023-12-12T09:57:28.370697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:50:04.248976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367930+0000","last_change":"2023-12-12T09:56:38.276052+0000","last_active":"2023-12-12T09:57:28.367930+0000","last_peered":"2023-12-12T09:57:28.367930+0000","last_clean":"2023-12-12T09:57:28.367930+0000","last_became_active":"2023-12-12T09:56:38.275712+0000","last_became_peered":"2023-12-12T09:56:38.275712+0000","last_unstale":"2023-12-12T09:57:28.367930+0000","last_undegraded":"2023-12-12T09:57:28.367930+0000","last_fullsized":"2023-12-12T09:57:28.367930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:37:06.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371935+0000","last_change":"2023-12-12T09:56:38.284052+0000","last_active":"2023-12-12T09:57:28.371935+0000","last_peered":"2023-12-12T09:57:28.371935+0000","last_clean":"2023-12-12T09:57:28.371935+0000","last_became_active":"2023-12-12T09:56:38.283741+0000","last_became_peered":"2023-12-12T09:56:38.283741+0000","last_unstale":"2023-12-12T09:57:28.371935+0000","last_undegraded":"2023-12-12T09:57:28.371935+0000","last_fullsized":"2023-12-12T09:57:28.371935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:40:47.505972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370785+0000","last_change":"2023-12-12T09:56:38.281604+0000","last_active":"2023-12-12T09:57:28.370785+0000","last_peered":"2023-12-12T09:57:28.370785+0000","last_clean":"2023-12-12T09:57:28.370785+0000","last_became_active":"2023-12-12T09:56:38.281247+0000","last_became_peered":"2023-12-12T09:56:38.281247+0000","last_unstale":"2023-12-12T09:57:28.370785+0000","last_undegraded":"2023-12-12T09:57:28.370785+0000","last_fullsized":"2023-12-12T09:57:28.370785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:36:14.002676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370782+0000","last_change":"2023-12-12T09:56:38.273599+0000","last_active":"2023-12-12T09:57:28.370782+0000","last_peered":"2023-12-12T09:57:28.370782+0000","last_clean":"2023-12-12T09:57:28.370782+0000","last_became_active":"2023-12-12T09:56:38.273290+0000","last_became_peered":"2023-12-12T09:56:38.273290+0000","last_unstale":"2023-12-12T09:57:28.370782+0000","last_undegraded":"2023-12-12T09:57:28.370782+0000","last_fullsized":"2023-12-12T09:57:28.370782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:54:52.088314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370250+0000","last_change":"2023-12-12T09:56:38.275663+0000","last_active":"2023-12-12T09:57:28.370250+0000","last_peered":"2023-12-12T09:57:28.370250+0000","last_clean":"2023-12-12T09:57:28.370250+0000","last_became_active":"2023-12-12T09:56:38.275322+0000","last_became_peered":"2023-12-12T09:56:38.275322+0000","last_unstale":"2023-12-12T09:57:28.370250+0000","last_undegraded":"2023-12-12T09:57:28.370250+0000","last_fullsized":"2023-12-12T09:57:28.370250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:46:50.067328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366661+0000","last_change":"2023-12-12T09:56:38.274064+0000","last_active":"2023-12-12T09:57:28.366661+0000","last_peered":"2023-12-12T09:57:28.366661+0000","last_clean":"2023-12-12T09:57:28.366661+0000","last_became_active":"2023-12-12T09:56:38.273733+0000","last_became_peered":"2023-12-12T09:56:38.273733+0000","last_unstale":"2023-12-12T09:57:28.366661+0000","last_undegraded":"2023-12-12T09:57:28.366661+0000","last_fullsized":"2023-12-12T09:57:28.366661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:20:31.662432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366114+0000","last_change":"2023-12-12T09:56:38.277627+0000","last_active":"2023-12-12T09:57:28.366114+0000","last_peered":"2023-12-12T09:57:28.366114+0000","last_clean":"2023-12-12T09:57:28.366114+0000","last_became_active":"2023-12-12T09:56:38.277275+0000","last_became_peered":"2023-12-12T09:56:38.277275+0000","last_unstale":"2023-12-12T09:57:28.366114+0000","last_undegraded":"2023-12-12T09:57:28.366114+0000","last_fullsized":"2023-12-12T09:57:28.366114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:37:57.256691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370988+0000","last_change":"2023-12-12T09:56:38.276900+0000","last_active":"2023-12-12T09:57:28.370988+0000","last_peered":"2023-12-12T09:57:28.370988+0000","last_clean":"2023-12-12T09:57:28.370988+0000","last_became_active":"2023-12-12T09:56:38.276384+0000","last_became_peered":"2023-12-12T09:56:38.276384+0000","last_unstale":"2023-12-12T09:57:28.370988+0000","last_undegraded":"2023-12-12T09:57:28.370988+0000","last_fullsized":"2023-12-12T09:57:28.370988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:13:37.097349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370598+0000","last_change":"2023-12-12T09:56:42.509239+0000","last_active":"2023-12-12T09:57:28.370598+0000","last_peered":"2023-12-12T09:57:28.370598+0000","last_clean":"2023-12-12T09:57:28.370598+0000","last_became_active":"2023-12-12T09:56:42.508699+0000","last_became_peered":"2023-12-12T09:56:42.508699+0000","last_unstale":"2023-12-12T09:57:28.370598+0000","last_undegraded":"2023-12-12T09:57:28.370598+0000","last_fullsized":"2023-12-12T09:57:28.370598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:12:39.501023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:43.367577+0000","last_change":"2023-12-12T09:56:37.476258+0000","last_active":"2023-12-12T09:57:43.367577+0000","last_peered":"2023-12-12T09:57:43.367577+0000","last_clean":"2023-12-12T09:57:43.367577+0000","last_became_active":"2023-12-12T09:56:37.475886+0000","last_became_peered":"2023-12-12T09:56:37.475886+0000","last_unstale":"2023-12-12T09:57:43.367577+0000","last_undegraded":"2023-12-12T09:57:43.367577+0000","last_fullsized":"2023-12-12T09:57:43.367577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:29:34.770549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370192+0000","last_change":"2023-12-12T09:56:38.284968+0000","last_active":"2023-12-12T09:57:28.370192+0000","last_peered":"2023-12-12T09:57:28.370192+0000","last_clean":"2023-12-12T09:57:28.370192+0000","last_became_active":"2023-12-12T09:56:38.284661+0000","last_became_peered":"2023-12-12T09:56:38.284661+0000","last_unstale":"2023-12-12T09:57:28.370192+0000","last_undegraded":"2023-12-12T09:57:28.370192+0000","last_fullsized":"2023-12-12T09:57:28.370192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:27:02.489220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.565229+0000","last_change":"2023-12-12T09:57:29.565229+0000","last_active":"2023-12-12T09:57:29.565229+0000","last_peered":"2023-12-12T09:57:29.565229+0000","last_clean":"2023-12-12T09:57:29.565229+0000","last_became_active":"2023-12-12T09:57:29.564821+0000","last_became_peered":"2023-12-12T09:57:29.564821+0000","last_unstale":"2023-12-12T09:57:29.565229+0000","last_undegraded":"2023-12-12T09:57:29.565229+0000","last_fullsized":"2023-12-12T09:57:29.565229+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:42:03.823715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370297+0000","last_change":"2023-12-12T09:56:37.471047+0000","last_active":"2023-12-12T09:57:28.370297+0000","last_peered":"2023-12-12T09:57:28.370297+0000","last_clean":"2023-12-12T09:57:28.370297+0000","last_became_active":"2023-12-12T09:56:37.470689+0000","last_became_peered":"2023-12-12T09:56:37.470689+0000","last_unstale":"2023-12-12T09:57:28.370297+0000","last_undegraded":"2023-12-12T09:57:28.370297+0000","last_fullsized":"2023-12-12T09:57:28.370297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:00.648165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371490+0000","last_change":"2023-12-12T09:56:38.275884+0000","last_active":"2023-12-12T09:57:28.371490+0000","last_peered":"2023-12-12T09:57:28.371490+0000","last_clean":"2023-12-12T09:57:28.371490+0000","last_became_active":"2023-12-12T09:56:38.275161+0000","last_became_peered":"2023-12-12T09:56:38.275161+0000","last_unstale":"2023-12-12T09:57:28.371490+0000","last_undegraded":"2023-12-12T09:57:28.371490+0000","last_fullsized":"2023-12-12T09:57:28.371490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:06:02.863719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367646+0000","last_change":"2023-12-12T09:56:42.507539+0000","last_active":"2023-12-12T09:57:28.367646+0000","last_peered":"2023-12-12T09:57:28.367646+0000","last_clean":"2023-12-12T09:57:28.367646+0000","last_became_active":"2023-12-12T09:56:42.507008+0000","last_became_peered":"2023-12-12T09:56:42.507008+0000","last_unstale":"2023-12-12T09:57:28.367646+0000","last_undegraded":"2023-12-12T09:57:28.367646+0000","last_fullsized":"2023-12-12T09:57:28.367646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:28:30.449096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367956+0000","last_change":"2023-12-12T09:56:37.475002+0000","last_active":"2023-12-12T09:57:28.367956+0000","last_peered":"2023-12-12T09:57:28.367956+0000","last_clean":"2023-12-12T09:57:28.367956+0000","last_became_active":"2023-12-12T09:56:37.474675+0000","last_became_peered":"2023-12-12T09:56:37.474675+0000","last_unstale":"2023-12-12T09:57:28.367956+0000","last_undegraded":"2023-12-12T09:57:28.367956+0000","last_fullsized":"2023-12-12T09:57:28.367956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:38:15.880299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371220+0000","last_change":"2023-12-12T09:56:38.280532+0000","last_active":"2023-12-12T09:57:28.371220+0000","last_peered":"2023-12-12T09:57:28.371220+0000","last_clean":"2023-12-12T09:57:28.371220+0000","last_became_active":"2023-12-12T09:56:38.280177+0000","last_became_peered":"2023-12-12T09:56:38.280177+0000","last_unstale":"2023-12-12T09:57:28.371220+0000","last_undegraded":"2023-12-12T09:57:28.371220+0000","last_fullsized":"2023-12-12T09:57:28.371220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:02:34.818978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367543+0000","last_change":"2023-12-12T09:56:42.504506+0000","last_active":"2023-12-12T09:57:28.367543+0000","last_peered":"2023-12-12T09:57:28.367543+0000","last_clean":"2023-12-12T09:57:28.367543+0000","last_became_active":"2023-12-12T09:56:42.504104+0000","last_became_peered":"2023-12-12T09:56:42.504104+0000","last_unstale":"2023-12-12T09:57:28.367543+0000","last_undegraded":"2023-12-12T09:57:28.367543+0000","last_fullsized":"2023-12-12T09:57:28.367543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:12:41.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367471+0000","last_change":"2023-12-12T09:56:37.475222+0000","last_active":"2023-12-12T09:57:28.367471+0000","last_peered":"2023-12-12T09:57:28.367471+0000","last_clean":"2023-12-12T09:57:28.367471+0000","last_became_active":"2023-12-12T09:56:37.474917+0000","last_became_peered":"2023-12-12T09:56:37.474917+0000","last_unstale":"2023-12-12T09:57:28.367471+0000","last_undegraded":"2023-12-12T09:57:28.367471+0000","last_fullsized":"2023-12-12T09:57:28.367471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:26:36.639215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.565521+0000","last_change":"2023-12-12T09:57:29.565521+0000","last_active":"2023-12-12T09:57:29.565521+0000","last_peered":"2023-12-12T09:57:29.565521+0000","last_clean":"2023-12-12T09:57:29.565521+0000","last_became_active":"2023-12-12T09:57:29.565139+0000","last_became_peered":"2023-12-12T09:57:29.565139+0000","last_unstale":"2023-12-12T09:57:29.565521+0000","last_undegraded":"2023-12-12T09:57:29.565521+0000","last_fullsized":"2023-12-12T09:57:29.565521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:54:53.819229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370770+0000","last_change":"2023-12-12T09:56:42.502196+0000","last_active":"2023-12-12T09:57:28.370770+0000","last_peered":"2023-12-12T09:57:28.370770+0000","last_clean":"2023-12-12T09:57:28.370770+0000","last_became_active":"2023-12-12T09:56:42.501789+0000","last_became_peered":"2023-12-12T09:56:42.501789+0000","last_unstale":"2023-12-12T09:57:28.370770+0000","last_undegraded":"2023-12-12T09:57:28.370770+0000","last_fullsized":"2023-12-12T09:57:28.370770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:04:02.656074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":77,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:41.396579+0000","last_change":"2023-12-12T09:57:29.386574+0000","last_active":"2023-12-12T09:57:41.396579+0000","last_peered":"2023-12-12T09:57:41.396579+0000","last_clean":"2023-12-12T09:57:41.396579+0000","last_became_active":"2023-12-12T09:57:29.378495+0000","last_became_peered":"2023-12-12T09:57:29.378495+0000","last_unstale":"2023-12-12T09:57:41.396579+0000","last_undegraded":"2023-12-12T09:57:41.396579+0000","last_fullsized":"2023-12-12T09:57:41.396579+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:47:46.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.368536+0000","last_change":"2023-12-12T09:56:38.281152+0000","last_active":"2023-12-12T09:57:28.368536+0000","last_peered":"2023-12-12T09:57:28.368536+0000","last_clean":"2023-12-12T09:57:28.368536+0000","last_became_active":"2023-12-12T09:56:38.280793+0000","last_became_peered":"2023-12-12T09:56:38.280793+0000","last_unstale":"2023-12-12T09:57:28.368536+0000","last_undegraded":"2023-12-12T09:57:28.368536+0000","last_fullsized":"2023-12-12T09:57:28.368536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370821+0000","last_change":"2023-12-12T09:56:42.507439+0000","last_active":"2023-12-12T09:57:28.370821+0000","last_peered":"2023-12-12T09:57:28.370821+0000","last_clean":"2023-12-12T09:57:28.370821+0000","last_became_active":"2023-12-12T09:56:42.506980+0000","last_became_peered":"2023-12-12T09:56:42.506980+0000","last_unstale":"2023-12-12T09:57:28.370821+0000","last_undegraded":"2023-12-12T09:57:28.370821+0000","last_fullsized":"2023-12-12T09:57:28.370821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:37.086804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365116+0000","last_change":"2023-12-12T09:56:37.473666+0000","last_active":"2023-12-12T09:57:28.365116+0000","last_peered":"2023-12-12T09:57:28.365116+0000","last_clean":"2023-12-12T09:57:28.365116+0000","last_became_active":"2023-12-12T09:56:37.473349+0000","last_became_peered":"2023-12-12T09:56:37.473349+0000","last_unstale":"2023-12-12T09:57:28.365116+0000","last_undegraded":"2023-12-12T09:57:28.365116+0000","last_fullsized":"2023-12-12T09:57:28.365116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.590517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.373931+0000","last_change":"2023-12-12T09:56:38.279220+0000","last_active":"2023-12-12T09:57:28.373931+0000","last_peered":"2023-12-12T09:57:28.373931+0000","last_clean":"2023-12-12T09:57:28.373931+0000","last_became_active":"2023-12-12T09:56:38.278950+0000","last_became_peered":"2023-12-12T09:56:38.278950+0000","last_unstale":"2023-12-12T09:57:28.373931+0000","last_undegraded":"2023-12-12T09:57:28.373931+0000","last_fullsized":"2023-12-12T09:57:28.373931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:06:24.247294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366660+0000","last_change":"2023-12-12T09:56:42.507022+0000","last_active":"2023-12-12T09:57:28.366660+0000","last_peered":"2023-12-12T09:57:28.366660+0000","last_clean":"2023-12-12T09:57:28.366660+0000","last_became_active":"2023-12-12T09:56:42.506575+0000","last_became_peered":"2023-12-12T09:56:42.506575+0000","last_unstale":"2023-12-12T09:57:28.366660+0000","last_undegraded":"2023-12-12T09:57:28.366660+0000","last_fullsized":"2023-12-12T09:57:28.366660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:46.685081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371824+0000","last_change":"2023-12-12T09:56:37.464083+0000","last_active":"2023-12-12T09:57:28.371824+0000","last_peered":"2023-12-12T09:57:28.371824+0000","last_clean":"2023-12-12T09:57:28.371824+0000","last_became_active":"2023-12-12T09:56:37.463774+0000","last_became_peered":"2023-12-12T09:56:37.463774+0000","last_unstale":"2023-12-12T09:57:28.371824+0000","last_undegraded":"2023-12-12T09:57:28.371824+0000","last_fullsized":"2023-12-12T09:57:28.371824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:34.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370073+0000","last_change":"2023-12-12T09:56:38.278729+0000","last_active":"2023-12-12T09:57:28.370073+0000","last_peered":"2023-12-12T09:57:28.370073+0000","last_clean":"2023-12-12T09:57:28.370073+0000","last_became_active":"2023-12-12T09:56:38.278193+0000","last_became_peered":"2023-12-12T09:56:38.278193+0000","last_unstale":"2023-12-12T09:57:28.370073+0000","last_undegraded":"2023-12-12T09:57:28.370073+0000","last_fullsized":"2023-12-12T09:57:28.370073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:20:25.854114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371142+0000","last_change":"2023-12-12T09:56:42.504228+0000","last_active":"2023-12-12T09:57:28.371142+0000","last_peered":"2023-12-12T09:57:28.371142+0000","last_clean":"2023-12-12T09:57:28.371142+0000","last_became_active":"2023-12-12T09:56:42.503823+0000","last_became_peered":"2023-12-12T09:56:42.503823+0000","last_unstale":"2023-12-12T09:57:28.371142+0000","last_undegraded":"2023-12-12T09:57:28.371142+0000","last_fullsized":"2023-12-12T09:57:28.371142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:54:05.263784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371368+0000","last_change":"2023-12-12T09:56:37.472108+0000","last_active":"2023-12-12T09:57:28.371368+0000","last_peered":"2023-12-12T09:57:28.371368+0000","last_clean":"2023-12-12T09:57:28.371368+0000","last_became_active":"2023-12-12T09:56:37.471822+0000","last_became_peered":"2023-12-12T09:56:37.471822+0000","last_unstale":"2023-12-12T09:57:28.371368+0000","last_undegraded":"2023-12-12T09:57:28.371368+0000","last_fullsized":"2023-12-12T09:57:28.371368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:19:24.432739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365200+0000","last_change":"2023-12-12T09:56:38.274724+0000","last_active":"2023-12-12T09:57:28.365200+0000","last_peered":"2023-12-12T09:57:28.365200+0000","last_clean":"2023-12-12T09:57:28.365200+0000","last_became_active":"2023-12-12T09:56:38.274383+0000","last_became_peered":"2023-12-12T09:56:38.274383+0000","last_unstale":"2023-12-12T09:57:28.365200+0000","last_undegraded":"2023-12-12T09:57:28.365200+0000","last_fullsized":"2023-12-12T09:57:28.365200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:21:37.512880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370055+0000","last_change":"2023-12-12T09:56:42.505510+0000","last_active":"2023-12-12T09:57:28.370055+0000","last_peered":"2023-12-12T09:57:28.370055+0000","last_clean":"2023-12-12T09:57:28.370055+0000","last_became_active":"2023-12-12T09:56:42.504141+0000","last_became_peered":"2023-12-12T09:56:42.504141+0000","last_unstale":"2023-12-12T09:57:28.370055+0000","last_undegraded":"2023-12-12T09:57:28.370055+0000","last_fullsized":"2023-12-12T09:57:28.370055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:48:00.108207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":97,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:43.378205+0000","last_change":"2023-12-12T09:56:37.466706+0000","last_active":"2023-12-12T09:57:43.378205+0000","last_peered":"2023-12-12T09:57:43.378205+0000","last_clean":"2023-12-12T09:57:43.378205+0000","last_became_active":"2023-12-12T09:56:37.466268+0000","last_became_peered":"2023-12-12T09:56:37.466268+0000","last_unstale":"2023-12-12T09:57:43.378205+0000","last_undegraded":"2023-12-12T09:57:43.378205+0000","last_fullsized":"2023-12-12T09:57:43.378205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:54:50.868616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371688+0000","last_change":"2023-12-12T09:56:38.277134+0000","last_active":"2023-12-12T09:57:28.371688+0000","last_peered":"2023-12-12T09:57:28.371688+0000","last_clean":"2023-12-12T09:57:28.371688+0000","last_became_active":"2023-12-12T09:56:38.276701+0000","last_became_peered":"2023-12-12T09:56:38.276701+0000","last_unstale":"2023-12-12T09:57:28.371688+0000","last_undegraded":"2023-12-12T09:57:28.371688+0000","last_fullsized":"2023-12-12T09:57:28.371688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:46:02.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365168+0000","last_change":"2023-12-12T09:56:42.506494+0000","last_active":"2023-12-12T09:57:28.365168+0000","last_peered":"2023-12-12T09:57:28.365168+0000","last_clean":"2023-12-12T09:57:28.365168+0000","last_became_active":"2023-12-12T09:56:42.505965+0000","last_became_peered":"2023-12-12T09:56:42.505965+0000","last_unstale":"2023-12-12T09:57:28.365168+0000","last_undegraded":"2023-12-12T09:57:28.365168+0000","last_fullsized":"2023-12-12T09:57:28.365168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:32:01.294474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371457+0000","last_change":"2023-12-12T09:56:37.469046+0000","last_active":"2023-12-12T09:57:28.371457+0000","last_peered":"2023-12-12T09:57:28.371457+0000","last_clean":"2023-12-12T09:57:28.371457+0000","last_became_active":"2023-12-12T09:56:37.468112+0000","last_became_peered":"2023-12-12T09:56:37.468112+0000","last_unstale":"2023-12-12T09:57:28.371457+0000","last_undegraded":"2023-12-12T09:57:28.371457+0000","last_fullsized":"2023-12-12T09:57:28.371457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:39:35.356243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371363+0000","last_change":"2023-12-12T09:56:38.280000+0000","last_active":"2023-12-12T09:57:28.371363+0000","last_peered":"2023-12-12T09:57:28.371363+0000","last_clean":"2023-12-12T09:57:28.371363+0000","last_became_active":"2023-12-12T09:56:38.279723+0000","last_became_peered":"2023-12-12T09:56:38.279723+0000","last_unstale":"2023-12-12T09:57:28.371363+0000","last_undegraded":"2023-12-12T09:57:28.371363+0000","last_fullsized":"2023-12-12T09:57:28.371363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:56:35.326614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367754+0000","last_change":"2023-12-12T09:56:42.506820+0000","last_active":"2023-12-12T09:57:28.367754+0000","last_peered":"2023-12-12T09:57:28.367754+0000","last_clean":"2023-12-12T09:57:28.367754+0000","last_became_active":"2023-12-12T09:56:42.506481+0000","last_became_peered":"2023-12-12T09:56:42.506481+0000","last_unstale":"2023-12-12T09:57:28.367754+0000","last_undegraded":"2023-12-12T09:57:28.367754+0000","last_fullsized":"2023-12-12T09:57:28.367754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:10:50.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372940+0000","last_change":"2023-12-12T09:56:37.470283+0000","last_active":"2023-12-12T09:57:28.372940+0000","last_peered":"2023-12-12T09:57:28.372940+0000","last_clean":"2023-12-12T09:57:28.372940+0000","last_became_active":"2023-12-12T09:56:37.469568+0000","last_became_peered":"2023-12-12T09:56:37.469568+0000","last_unstale":"2023-12-12T09:57:28.372940+0000","last_undegraded":"2023-12-12T09:57:28.372940+0000","last_fullsized":"2023-12-12T09:57:28.372940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:24.427658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371626+0000","last_change":"2023-12-12T09:56:38.279180+0000","last_active":"2023-12-12T09:57:28.371626+0000","last_peered":"2023-12-12T09:57:28.371626+0000","last_clean":"2023-12-12T09:57:28.371626+0000","last_became_active":"2023-12-12T09:56:38.278493+0000","last_became_peered":"2023-12-12T09:56:38.278493+0000","last_unstale":"2023-12-12T09:57:28.371626+0000","last_undegraded":"2023-12-12T09:57:28.371626+0000","last_fullsized":"2023-12-12T09:57:28.371626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:17:52.468663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365640+0000","last_change":"2023-12-12T09:56:42.505140+0000","last_active":"2023-12-12T09:57:28.365640+0000","last_peered":"2023-12-12T09:57:28.365640+0000","last_clean":"2023-12-12T09:57:28.365640+0000","last_became_active":"2023-12-12T09:56:42.504655+0000","last_became_peered":"2023-12-12T09:56:42.504655+0000","last_unstale":"2023-12-12T09:57:28.365640+0000","last_undegraded":"2023-12-12T09:57:28.365640+0000","last_fullsized":"2023-12-12T09:57:28.365640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:58:18.004084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":135,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:43.397643+0000","last_change":"2023-12-12T09:56:37.468048+0000","last_active":"2023-12-12T09:57:43.397643+0000","last_peered":"2023-12-12T09:57:43.397643+0000","last_clean":"2023-12-12T09:57:43.397643+0000","last_became_active":"2023-12-12T09:56:37.467744+0000","last_became_peered":"2023-12-12T09:56:37.467744+0000","last_unstale":"2023-12-12T09:57:43.397643+0000","last_undegraded":"2023-12-12T09:57:43.397643+0000","last_fullsized":"2023-12-12T09:57:43.397643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:29:56.033230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59852,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":81,"num_read_kb":99,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367669+0000","last_change":"2023-12-12T09:56:38.284640+0000","last_active":"2023-12-12T09:57:28.367669+0000","last_peered":"2023-12-12T09:57:28.367669+0000","last_clean":"2023-12-12T09:57:28.367669+0000","last_became_active":"2023-12-12T09:56:38.284351+0000","last_became_peered":"2023-12-12T09:56:38.284351+0000","last_unstale":"2023-12-12T09:57:28.367669+0000","last_undegraded":"2023-12-12T09:57:28.367669+0000","last_fullsized":"2023-12-12T09:57:28.367669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:22.910616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.373189+0000","last_change":"2023-12-12T09:56:42.502821+0000","last_active":"2023-12-12T09:57:28.373189+0000","last_peered":"2023-12-12T09:57:28.373189+0000","last_clean":"2023-12-12T09:57:28.373189+0000","last_became_active":"2023-12-12T09:56:42.502377+0000","last_became_peered":"2023-12-12T09:56:42.502377+0000","last_unstale":"2023-12-12T09:57:28.373189+0000","last_undegraded":"2023-12-12T09:57:28.373189+0000","last_fullsized":"2023-12-12T09:57:28.373189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:41:22.913984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370931+0000","last_change":"2023-12-12T09:56:37.468656+0000","last_active":"2023-12-12T09:57:28.370931+0000","last_peered":"2023-12-12T09:57:28.370931+0000","last_clean":"2023-12-12T09:57:28.370931+0000","last_became_active":"2023-12-12T09:56:37.468353+0000","last_became_peered":"2023-12-12T09:56:37.468353+0000","last_unstale":"2023-12-12T09:57:28.370931+0000","last_undegraded":"2023-12-12T09:57:28.370931+0000","last_fullsized":"2023-12-12T09:57:28.370931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:51:21.477423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371693+0000","last_change":"2023-12-12T09:56:38.277290+0000","last_active":"2023-12-12T09:57:28.371693+0000","last_peered":"2023-12-12T09:57:28.371693+0000","last_clean":"2023-12-12T09:57:28.371693+0000","last_became_active":"2023-12-12T09:56:38.276972+0000","last_became_peered":"2023-12-12T09:56:38.276972+0000","last_unstale":"2023-12-12T09:57:28.371693+0000","last_undegraded":"2023-12-12T09:57:28.371693+0000","last_fullsized":"2023-12-12T09:57:28.371693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:03:25.254368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365025+0000","last_change":"2023-12-12T09:56:42.506900+0000","last_active":"2023-12-12T09:57:28.365025+0000","last_peered":"2023-12-12T09:57:28.365025+0000","last_clean":"2023-12-12T09:57:28.365025+0000","last_became_active":"2023-12-12T09:56:42.506470+0000","last_became_peered":"2023-12-12T09:56:42.506470+0000","last_unstale":"2023-12-12T09:57:28.365025+0000","last_undegraded":"2023-12-12T09:57:28.365025+0000","last_fullsized":"2023-12-12T09:57:28.365025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:04:17.359766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371449+0000","last_change":"2023-12-12T09:56:37.472598+0000","last_active":"2023-12-12T09:57:28.371449+0000","last_peered":"2023-12-12T09:57:28.371449+0000","last_clean":"2023-12-12T09:57:28.371449+0000","last_became_active":"2023-12-12T09:56:37.471809+0000","last_became_peered":"2023-12-12T09:56:37.471809+0000","last_unstale":"2023-12-12T09:57:28.371449+0000","last_undegraded":"2023-12-12T09:57:28.371449+0000","last_fullsized":"2023-12-12T09:57:28.371449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:54:08.677081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370962+0000","last_change":"2023-12-12T09:56:38.277180+0000","last_active":"2023-12-12T09:57:28.370962+0000","last_peered":"2023-12-12T09:57:28.370962+0000","last_clean":"2023-12-12T09:57:28.370962+0000","last_became_active":"2023-12-12T09:56:38.276878+0000","last_became_peered":"2023-12-12T09:56:38.276878+0000","last_unstale":"2023-12-12T09:57:28.370962+0000","last_undegraded":"2023-12-12T09:57:28.370962+0000","last_fullsized":"2023-12-12T09:57:28.370962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:18:49.704820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.435169+0000","last_change":"2023-12-12T09:57:29.434750+0000","last_active":"2023-12-12T09:57:29.435169+0000","last_peered":"2023-12-12T09:57:29.435169+0000","last_clean":"2023-12-12T09:57:29.435169+0000","last_became_active":"2023-12-12T09:57:29.434302+0000","last_became_peered":"2023-12-12T09:57:29.434302+0000","last_unstale":"2023-12-12T09:57:29.435169+0000","last_undegraded":"2023-12-12T09:57:29.435169+0000","last_fullsized":"2023-12-12T09:57:29.435169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:51:43.809887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:40.352782+0000","last_change":"2023-12-12T09:56:37.468329+0000","last_active":"2023-12-12T09:57:40.352782+0000","last_peered":"2023-12-12T09:57:40.352782+0000","last_clean":"2023-12-12T09:57:40.352782+0000","last_became_active":"2023-12-12T09:56:37.467886+0000","last_became_peered":"2023-12-12T09:56:37.467886+0000","last_unstale":"2023-12-12T09:57:40.352782+0000","last_undegraded":"2023-12-12T09:57:40.352782+0000","last_fullsized":"2023-12-12T09:57:40.352782+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:23:35.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370135+0000","last_change":"2023-12-12T09:56:38.281280+0000","last_active":"2023-12-12T09:57:28.370135+0000","last_peered":"2023-12-12T09:57:28.370135+0000","last_clean":"2023-12-12T09:57:28.370135+0000","last_became_active":"2023-12-12T09:56:38.280993+0000","last_became_peered":"2023-12-12T09:56:38.280993+0000","last_unstale":"2023-12-12T09:57:28.370135+0000","last_undegraded":"2023-12-12T09:57:28.370135+0000","last_fullsized":"2023-12-12T09:57:28.370135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:53:44.562981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372891+0000","last_change":"2023-12-12T09:56:42.508920+0000","last_active":"2023-12-12T09:57:28.372891+0000","last_peered":"2023-12-12T09:57:28.372891+0000","last_clean":"2023-12-12T09:57:28.372891+0000","last_became_active":"2023-12-12T09:56:42.508491+0000","last_became_peered":"2023-12-12T09:56:42.508491+0000","last_unstale":"2023-12-12T09:57:28.372891+0000","last_undegraded":"2023-12-12T09:57:28.372891+0000","last_fullsized":"2023-12-12T09:57:28.372891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:12:50.596137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.395734+0000","last_change":"2023-12-12T09:57:29.395375+0000","last_active":"2023-12-12T09:57:29.395734+0000","last_peered":"2023-12-12T09:57:29.395734+0000","last_clean":"2023-12-12T09:57:29.395734+0000","last_became_active":"2023-12-12T09:57:29.380667+0000","last_became_peered":"2023-12-12T09:57:29.380667+0000","last_unstale":"2023-12-12T09:57:29.395734+0000","last_undegraded":"2023-12-12T09:57:29.395734+0000","last_fullsized":"2023-12-12T09:57:29.395734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:16:04.815137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.381148+0000","last_change":"2023-12-12T09:57:29.381148+0000","last_active":"2023-12-12T09:57:29.381148+0000","last_peered":"2023-12-12T09:57:29.381148+0000","last_clean":"2023-12-12T09:57:29.381148+0000","last_became_active":"2023-12-12T09:57:29.380710+0000","last_became_peered":"2023-12-12T09:57:29.380710+0000","last_unstale":"2023-12-12T09:57:29.381148+0000","last_undegraded":"2023-12-12T09:57:29.381148+0000","last_fullsized":"2023-12-12T09:57:29.381148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:05:45.295141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.373394+0000","last_change":"2023-12-12T09:56:42.499828+0000","last_active":"2023-12-12T09:57:28.373394+0000","last_peered":"2023-12-12T09:57:28.373394+0000","last_clean":"2023-12-12T09:57:28.373394+0000","last_became_active":"2023-12-12T09:56:42.499453+0000","last_became_peered":"2023-12-12T09:56:42.499453+0000","last_unstale":"2023-12-12T09:57:28.373394+0000","last_undegraded":"2023-12-12T09:57:28.373394+0000","last_fullsized":"2023-12-12T09:57:28.373394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:20:12.609145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.384641+0000","last_change":"2023-12-12T09:57:29.384213+0000","last_active":"2023-12-12T09:57:29.384641+0000","last_peered":"2023-12-12T09:57:29.384641+0000","last_clean":"2023-12-12T09:57:29.384641+0000","last_became_active":"2023-12-12T09:57:29.378849+0000","last_became_peered":"2023-12-12T09:57:29.378849+0000","last_unstale":"2023-12-12T09:57:29.384641+0000","last_undegraded":"2023-12-12T09:57:29.384641+0000","last_fullsized":"2023-12-12T09:57:29.384641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:39:08.545539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365885+0000","last_change":"2023-12-12T09:56:38.281060+0000","last_active":"2023-12-12T09:57:28.365885+0000","last_peered":"2023-12-12T09:57:28.365885+0000","last_clean":"2023-12-12T09:57:28.365885+0000","last_became_active":"2023-12-12T09:56:38.280725+0000","last_became_peered":"2023-12-12T09:56:38.280725+0000","last_unstale":"2023-12-12T09:57:28.365885+0000","last_undegraded":"2023-12-12T09:57:28.365885+0000","last_fullsized":"2023-12-12T09:57:28.365885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:14.592213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.435128+0000","last_change":"2023-12-12T09:57:29.435128+0000","last_active":"2023-12-12T09:57:29.435128+0000","last_peered":"2023-12-12T09:57:29.435128+0000","last_clean":"2023-12-12T09:57:29.435128+0000","last_became_active":"2023-12-12T09:57:29.434622+0000","last_became_peered":"2023-12-12T09:57:29.434622+0000","last_unstale":"2023-12-12T09:57:29.435128+0000","last_undegraded":"2023-12-12T09:57:29.435128+0000","last_fullsized":"2023-12-12T09:57:29.435128+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:55.698693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365797+0000","last_change":"2023-12-12T09:56:37.465071+0000","last_active":"2023-12-12T09:57:28.365797+0000","last_peered":"2023-12-12T09:57:28.365797+0000","last_clean":"2023-12-12T09:57:28.365797+0000","last_became_active":"2023-12-12T09:56:37.464717+0000","last_became_peered":"2023-12-12T09:56:37.464717+0000","last_unstale":"2023-12-12T09:57:28.365797+0000","last_undegraded":"2023-12-12T09:57:28.365797+0000","last_fullsized":"2023-12-12T09:57:28.365797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:04:52.167228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365419+0000","last_change":"2023-12-12T09:56:38.273276+0000","last_active":"2023-12-12T09:57:28.365419+0000","last_peered":"2023-12-12T09:57:28.365419+0000","last_clean":"2023-12-12T09:57:28.365419+0000","last_became_active":"2023-12-12T09:56:38.272966+0000","last_became_peered":"2023-12-12T09:56:38.272966+0000","last_unstale":"2023-12-12T09:57:28.365419+0000","last_undegraded":"2023-12-12T09:57:28.365419+0000","last_fullsized":"2023-12-12T09:57:28.365419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:22.352843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371748+0000","last_change":"2023-12-12T09:56:42.510808+0000","last_active":"2023-12-12T09:57:28.371748+0000","last_peered":"2023-12-12T09:57:28.371748+0000","last_clean":"2023-12-12T09:57:28.371748+0000","last_became_active":"2023-12-12T09:56:42.510408+0000","last_became_peered":"2023-12-12T09:56:42.510408+0000","last_unstale":"2023-12-12T09:57:28.371748+0000","last_undegraded":"2023-12-12T09:57:28.371748+0000","last_fullsized":"2023-12-12T09:57:28.371748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:53:08.573345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:43.368321+0000","last_change":"2023-12-12T09:56:37.468863+0000","last_active":"2023-12-12T09:57:43.368321+0000","last_peered":"2023-12-12T09:57:43.368321+0000","last_clean":"2023-12-12T09:57:43.368321+0000","last_became_active":"2023-12-12T09:56:37.468221+0000","last_became_peered":"2023-12-12T09:56:37.468221+0000","last_unstale":"2023-12-12T09:57:43.368321+0000","last_undegraded":"2023-12-12T09:57:43.368321+0000","last_fullsized":"2023-12-12T09:57:43.368321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:39:17.559717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365273+0000","last_change":"2023-12-12T09:56:38.277983+0000","last_active":"2023-12-12T09:57:28.365273+0000","last_peered":"2023-12-12T09:57:28.365273+0000","last_clean":"2023-12-12T09:57:28.365273+0000","last_became_active":"2023-12-12T09:56:38.275926+0000","last_became_peered":"2023-12-12T09:56:38.275926+0000","last_unstale":"2023-12-12T09:57:28.365273+0000","last_undegraded":"2023-12-12T09:57:28.365273+0000","last_fullsized":"2023-12-12T09:57:28.365273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:57:51.474651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372852+0000","last_change":"2023-12-12T09:56:42.577662+0000","last_active":"2023-12-12T09:57:28.372852+0000","last_peered":"2023-12-12T09:57:28.372852+0000","last_clean":"2023-12-12T09:57:28.372852+0000","last_became_active":"2023-12-12T09:56:42.577188+0000","last_became_peered":"2023-12-12T09:56:42.577188+0000","last_unstale":"2023-12-12T09:57:28.372852+0000","last_undegraded":"2023-12-12T09:57:28.372852+0000","last_fullsized":"2023-12-12T09:57:28.372852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:58:35.341894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371562+0000","last_change":"2023-12-12T09:56:38.279537+0000","last_active":"2023-12-12T09:57:28.371562+0000","last_peered":"2023-12-12T09:57:28.371562+0000","last_clean":"2023-12-12T09:57:28.371562+0000","last_became_active":"2023-12-12T09:56:38.278739+0000","last_became_peered":"2023-12-12T09:56:38.278739+0000","last_unstale":"2023-12-12T09:57:28.371562+0000","last_undegraded":"2023-12-12T09:57:28.371562+0000","last_fullsized":"2023-12-12T09:57:28.371562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:35:05.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:42.354601+0000","last_change":"2023-12-12T09:56:37.465726+0000","last_active":"2023-12-12T09:57:42.354601+0000","last_peered":"2023-12-12T09:57:42.354601+0000","last_clean":"2023-12-12T09:57:42.354601+0000","last_became_active":"2023-12-12T09:56:37.465438+0000","last_became_peered":"2023-12-12T09:56:37.465438+0000","last_unstale":"2023-12-12T09:57:42.354601+0000","last_undegraded":"2023-12-12T09:57:42.354601+0000","last_fullsized":"2023-12-12T09:57:42.354601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:17:58.356893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370214+0000","last_change":"2023-12-12T09:56:42.505943+0000","last_active":"2023-12-12T09:57:28.370214+0000","last_peered":"2023-12-12T09:57:28.370214+0000","last_clean":"2023-12-12T09:57:28.370214+0000","last_became_active":"2023-12-12T09:56:42.505581+0000","last_became_peered":"2023-12-12T09:56:42.505581+0000","last_unstale":"2023-12-12T09:57:28.370214+0000","last_undegraded":"2023-12-12T09:57:28.370214+0000","last_fullsized":"2023-12-12T09:57:28.370214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:19:08.869868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369564+0000","last_change":"2023-12-12T09:56:38.284895+0000","last_active":"2023-12-12T09:57:28.369564+0000","last_peered":"2023-12-12T09:57:28.369564+0000","last_clean":"2023-12-12T09:57:28.369564+0000","last_became_active":"2023-12-12T09:56:38.284429+0000","last_became_peered":"2023-12-12T09:56:38.284429+0000","last_unstale":"2023-12-12T09:57:28.369564+0000","last_undegraded":"2023-12-12T09:57:28.369564+0000","last_fullsized":"2023-12-12T09:57:28.369564+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:14.170629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367718+0000","last_change":"2023-12-12T09:56:37.466748+0000","last_active":"2023-12-12T09:57:28.367718+0000","last_peered":"2023-12-12T09:57:28.367718+0000","last_clean":"2023-12-12T09:57:28.367718+0000","last_became_active":"2023-12-12T09:56:37.466459+0000","last_became_peered":"2023-12-12T09:56:37.466459+0000","last_unstale":"2023-12-12T09:57:28.367718+0000","last_undegraded":"2023-12-12T09:57:28.367718+0000","last_fullsized":"2023-12-12T09:57:28.367718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:21:43.819845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370077+0000","last_change":"2023-12-12T09:55:37.275186+0000","last_active":"2023-12-12T09:57:28.370077+0000","last_peered":"2023-12-12T09:57:28.370077+0000","last_clean":"2023-12-12T09:57:28.370077+0000","last_became_active":"2023-12-12T09:55:37.274808+0000","last_became_peered":"2023-12-12T09:55:37.274808+0000","last_unstale":"2023-12-12T09:57:28.370077+0000","last_undegraded":"2023-12-12T09:57:28.370077+0000","last_fullsized":"2023-12-12T09:57:28.370077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:42.502190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:42.502190+0000","last_clean_scrub_stamp":"2023-12-12T09:52:42.502190+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:27:08.942642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372950+0000","last_change":"2023-12-12T09:56:42.578219+0000","last_active":"2023-12-12T09:57:28.372950+0000","last_peered":"2023-12-12T09:57:28.372950+0000","last_clean":"2023-12-12T09:57:28.372950+0000","last_became_active":"2023-12-12T09:56:42.577589+0000","last_became_peered":"2023-12-12T09:56:42.577589+0000","last_unstale":"2023-12-12T09:57:28.372950+0000","last_undegraded":"2023-12-12T09:57:28.372950+0000","last_fullsized":"2023-12-12T09:57:28.372950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:40:54.162235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371463+0000","last_change":"2023-12-12T09:56:38.274163+0000","last_active":"2023-12-12T09:57:28.371463+0000","last_peered":"2023-12-12T09:57:28.371463+0000","last_clean":"2023-12-12T09:57:28.371463+0000","last_became_active":"2023-12-12T09:56:38.273837+0000","last_became_peered":"2023-12-12T09:56:38.273837+0000","last_unstale":"2023-12-12T09:57:28.371463+0000","last_undegraded":"2023-12-12T09:57:28.371463+0000","last_fullsized":"2023-12-12T09:57:28.371463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:46:53.550289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369921+0000","last_change":"2023-12-12T09:56:37.473275+0000","last_active":"2023-12-12T09:57:28.369921+0000","last_peered":"2023-12-12T09:57:28.369921+0000","last_clean":"2023-12-12T09:57:28.369921+0000","last_became_active":"2023-12-12T09:56:37.472973+0000","last_became_peered":"2023-12-12T09:56:37.472973+0000","last_unstale":"2023-12-12T09:57:28.369921+0000","last_undegraded":"2023-12-12T09:57:28.369921+0000","last_fullsized":"2023-12-12T09:57:28.369921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:03.660259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365011+0000","last_change":"2023-12-12T09:56:42.503299+0000","last_active":"2023-12-12T09:57:28.365011+0000","last_peered":"2023-12-12T09:57:28.365011+0000","last_clean":"2023-12-12T09:57:28.365011+0000","last_became_active":"2023-12-12T09:56:42.502862+0000","last_became_peered":"2023-12-12T09:56:42.502862+0000","last_unstale":"2023-12-12T09:57:28.365011+0000","last_undegraded":"2023-12-12T09:57:28.365011+0000","last_fullsized":"2023-12-12T09:57:28.365011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:47:14.211916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367788+0000","last_change":"2023-12-12T09:56:37.476084+0000","last_active":"2023-12-12T09:57:28.367788+0000","last_peered":"2023-12-12T09:57:28.367788+0000","last_clean":"2023-12-12T09:57:28.367788+0000","last_became_active":"2023-12-12T09:56:37.475760+0000","last_became_peered":"2023-12-12T09:56:37.475760+0000","last_unstale":"2023-12-12T09:57:28.367788+0000","last_undegraded":"2023-12-12T09:57:28.367788+0000","last_fullsized":"2023-12-12T09:57:28.367788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:30:43.487809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371845+0000","last_change":"2023-12-12T09:56:38.278502+0000","last_active":"2023-12-12T09:57:28.371845+0000","last_peered":"2023-12-12T09:57:28.371845+0000","last_clean":"2023-12-12T09:57:28.371845+0000","last_became_active":"2023-12-12T09:56:38.278210+0000","last_became_peered":"2023-12-12T09:56:38.278210+0000","last_unstale":"2023-12-12T09:57:28.371845+0000","last_undegraded":"2023-12-12T09:57:28.371845+0000","last_fullsized":"2023-12-12T09:57:28.371845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:25:09.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369978+0000","last_change":"2023-12-12T09:56:42.577598+0000","last_active":"2023-12-12T09:57:28.369978+0000","last_peered":"2023-12-12T09:57:28.369978+0000","last_clean":"2023-12-12T09:57:28.369978+0000","last_became_active":"2023-12-12T09:56:42.577147+0000","last_became_peered":"2023-12-12T09:56:42.577147+0000","last_unstale":"2023-12-12T09:57:28.369978+0000","last_undegraded":"2023-12-12T09:57:28.369978+0000","last_fullsized":"2023-12-12T09:57:28.369978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:05:11.279871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366141+0000","last_change":"2023-12-12T09:56:37.474862+0000","last_active":"2023-12-12T09:57:28.366141+0000","last_peered":"2023-12-12T09:57:28.366141+0000","last_clean":"2023-12-12T09:57:28.366141+0000","last_became_active":"2023-12-12T09:56:37.474463+0000","last_became_peered":"2023-12-12T09:56:37.474463+0000","last_unstale":"2023-12-12T09:57:28.366141+0000","last_undegraded":"2023-12-12T09:57:28.366141+0000","last_fullsized":"2023-12-12T09:57:28.366141+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:06.926221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370049+0000","last_change":"2023-12-12T09:56:38.279729+0000","last_active":"2023-12-12T09:57:28.370049+0000","last_peered":"2023-12-12T09:57:28.370049+0000","last_clean":"2023-12-12T09:57:28.370049+0000","last_became_active":"2023-12-12T09:56:38.277634+0000","last_became_peered":"2023-12-12T09:56:38.277634+0000","last_unstale":"2023-12-12T09:57:28.370049+0000","last_undegraded":"2023-12-12T09:57:28.370049+0000","last_fullsized":"2023-12-12T09:57:28.370049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:06:22.594507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370073+0000","last_change":"2023-12-12T09:56:42.502342+0000","last_active":"2023-12-12T09:57:28.370073+0000","last_peered":"2023-12-12T09:57:28.370073+0000","last_clean":"2023-12-12T09:57:28.370073+0000","last_became_active":"2023-12-12T09:56:42.501313+0000","last_became_peered":"2023-12-12T09:56:42.501313+0000","last_unstale":"2023-12-12T09:57:28.370073+0000","last_undegraded":"2023-12-12T09:57:28.370073+0000","last_fullsized":"2023-12-12T09:57:28.370073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:12:28.144029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.368295+0000","last_change":"2023-12-12T09:56:37.464428+0000","last_active":"2023-12-12T09:57:28.368295+0000","last_peered":"2023-12-12T09:57:28.368295+0000","last_clean":"2023-12-12T09:57:28.368295+0000","last_became_active":"2023-12-12T09:56:37.464159+0000","last_became_peered":"2023-12-12T09:56:37.464159+0000","last_unstale":"2023-12-12T09:57:28.368295+0000","last_undegraded":"2023-12-12T09:57:28.368295+0000","last_fullsized":"2023-12-12T09:57:28.368295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:14:53.779852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371245+0000","last_change":"2023-12-12T09:56:38.277023+0000","last_active":"2023-12-12T09:57:28.371245+0000","last_peered":"2023-12-12T09:57:28.371245+0000","last_clean":"2023-12-12T09:57:28.371245+0000","last_became_active":"2023-12-12T09:56:38.276708+0000","last_became_peered":"2023-12-12T09:56:38.276708+0000","last_unstale":"2023-12-12T09:57:28.371245+0000","last_undegraded":"2023-12-12T09:57:28.371245+0000","last_fullsized":"2023-12-12T09:57:28.371245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:08:28.244491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367939+0000","last_change":"2023-12-12T09:56:42.578025+0000","last_active":"2023-12-12T09:57:28.367939+0000","last_peered":"2023-12-12T09:57:28.367939+0000","last_clean":"2023-12-12T09:57:28.367939+0000","last_became_active":"2023-12-12T09:56:42.577570+0000","last_became_peered":"2023-12-12T09:56:42.577570+0000","last_unstale":"2023-12-12T09:57:28.367939+0000","last_undegraded":"2023-12-12T09:57:28.367939+0000","last_fullsized":"2023-12-12T09:57:28.367939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:51:02.998355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370426+0000","last_change":"2023-12-12T09:56:37.467815+0000","last_active":"2023-12-12T09:57:28.370426+0000","last_peered":"2023-12-12T09:57:28.370426+0000","last_clean":"2023-12-12T09:57:28.370426+0000","last_became_active":"2023-12-12T09:56:37.467110+0000","last_became_peered":"2023-12-12T09:56:37.467110+0000","last_unstale":"2023-12-12T09:57:28.370426+0000","last_undegraded":"2023-12-12T09:57:28.370426+0000","last_fullsized":"2023-12-12T09:57:28.370426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:40:24.325774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367205+0000","last_change":"2023-12-12T09:56:38.284770+0000","last_active":"2023-12-12T09:57:28.367205+0000","last_peered":"2023-12-12T09:57:28.367205+0000","last_clean":"2023-12-12T09:57:28.367205+0000","last_became_active":"2023-12-12T09:56:38.284459+0000","last_became_peered":"2023-12-12T09:56:38.284459+0000","last_unstale":"2023-12-12T09:57:28.367205+0000","last_undegraded":"2023-12-12T09:57:28.367205+0000","last_fullsized":"2023-12-12T09:57:28.367205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:02:42.787163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366174+0000","last_change":"2023-12-12T09:56:42.512278+0000","last_active":"2023-12-12T09:57:28.366174+0000","last_peered":"2023-12-12T09:57:28.366174+0000","last_clean":"2023-12-12T09:57:28.366174+0000","last_became_active":"2023-12-12T09:56:42.511598+0000","last_became_peered":"2023-12-12T09:56:42.511598+0000","last_unstale":"2023-12-12T09:57:28.366174+0000","last_undegraded":"2023-12-12T09:57:28.366174+0000","last_fullsized":"2023-12-12T09:57:28.366174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:00:03.571614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370149+0000","last_change":"2023-12-12T09:56:37.469274+0000","last_active":"2023-12-12T09:57:28.370149+0000","last_peered":"2023-12-12T09:57:28.370149+0000","last_clean":"2023-12-12T09:57:28.370149+0000","last_became_active":"2023-12-12T09:56:37.468947+0000","last_became_peered":"2023-12-12T09:56:37.468947+0000","last_unstale":"2023-12-12T09:57:28.370149+0000","last_undegraded":"2023-12-12T09:57:28.370149+0000","last_fullsized":"2023-12-12T09:57:28.370149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:26:25.559385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.364832+0000","last_change":"2023-12-12T09:56:38.274415+0000","last_active":"2023-12-12T09:57:28.364832+0000","last_peered":"2023-12-12T09:57:28.364832+0000","last_clean":"2023-12-12T09:57:28.364832+0000","last_became_active":"2023-12-12T09:56:38.274119+0000","last_became_peered":"2023-12-12T09:56:38.274119+0000","last_unstale":"2023-12-12T09:57:28.364832+0000","last_undegraded":"2023-12-12T09:57:28.364832+0000","last_fullsized":"2023-12-12T09:57:28.364832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:50.160355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367756+0000","last_change":"2023-12-12T09:56:42.506811+0000","last_active":"2023-12-12T09:57:28.367756+0000","last_peered":"2023-12-12T09:57:28.367756+0000","last_clean":"2023-12-12T09:57:28.367756+0000","last_became_active":"2023-12-12T09:56:42.506374+0000","last_became_peered":"2023-12-12T09:56:42.506374+0000","last_unstale":"2023-12-12T09:57:28.367756+0000","last_undegraded":"2023-12-12T09:57:28.367756+0000","last_fullsized":"2023-12-12T09:57:28.367756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:50:54.647403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370684+0000","last_change":"2023-12-12T09:56:37.473548+0000","last_active":"2023-12-12T09:57:28.370684+0000","last_peered":"2023-12-12T09:57:28.370684+0000","last_clean":"2023-12-12T09:57:28.370684+0000","last_became_active":"2023-12-12T09:56:37.473240+0000","last_became_peered":"2023-12-12T09:56:37.473240+0000","last_unstale":"2023-12-12T09:57:28.370684+0000","last_undegraded":"2023-12-12T09:57:28.370684+0000","last_fullsized":"2023-12-12T09:57:28.370684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:42:22.921437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369559+0000","last_change":"2023-12-12T09:56:38.276853+0000","last_active":"2023-12-12T09:57:28.369559+0000","last_peered":"2023-12-12T09:57:28.369559+0000","last_clean":"2023-12-12T09:57:28.369559+0000","last_became_active":"2023-12-12T09:56:38.276567+0000","last_became_peered":"2023-12-12T09:56:38.276567+0000","last_unstale":"2023-12-12T09:57:28.369559+0000","last_undegraded":"2023-12-12T09:57:28.369559+0000","last_fullsized":"2023-12-12T09:57:28.369559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:21:15.069389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367860+0000","last_change":"2023-12-12T09:56:42.579135+0000","last_active":"2023-12-12T09:57:28.367860+0000","last_peered":"2023-12-12T09:57:28.367860+0000","last_clean":"2023-12-12T09:57:28.367860+0000","last_became_active":"2023-12-12T09:56:42.578513+0000","last_became_peered":"2023-12-12T09:56:42.578513+0000","last_unstale":"2023-12-12T09:57:28.367860+0000","last_undegraded":"2023-12-12T09:57:28.367860+0000","last_fullsized":"2023-12-12T09:57:28.367860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:36:27.337524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.383906+0000","last_change":"2023-12-12T09:57:29.383906+0000","last_active":"2023-12-12T09:57:29.383906+0000","last_peered":"2023-12-12T09:57:29.383906+0000","last_clean":"2023-12-12T09:57:29.383906+0000","last_became_active":"2023-12-12T09:57:29.377499+0000","last_became_peered":"2023-12-12T09:57:29.377499+0000","last_unstale":"2023-12-12T09:57:29.383906+0000","last_undegraded":"2023-12-12T09:57:29.383906+0000","last_fullsized":"2023-12-12T09:57:29.383906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:53:37.904144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370945+0000","last_change":"2023-12-12T09:56:38.279661+0000","last_active":"2023-12-12T09:57:28.370945+0000","last_peered":"2023-12-12T09:57:28.370945+0000","last_clean":"2023-12-12T09:57:28.370945+0000","last_became_active":"2023-12-12T09:56:38.279330+0000","last_became_peered":"2023-12-12T09:56:38.279330+0000","last_unstale":"2023-12-12T09:57:28.370945+0000","last_undegraded":"2023-12-12T09:57:28.370945+0000","last_fullsized":"2023-12-12T09:57:28.370945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:19:15.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.565808+0000","last_change":"2023-12-12T09:57:29.565808+0000","last_active":"2023-12-12T09:57:29.565808+0000","last_peered":"2023-12-12T09:57:29.565808+0000","last_clean":"2023-12-12T09:57:29.565808+0000","last_became_active":"2023-12-12T09:57:29.565389+0000","last_became_peered":"2023-12-12T09:57:29.565389+0000","last_unstale":"2023-12-12T09:57:29.565808+0000","last_undegraded":"2023-12-12T09:57:29.565808+0000","last_fullsized":"2023-12-12T09:57:29.565808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:53:15.810984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":61,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:40.374324+0000","last_change":"2023-12-12T09:56:37.466958+0000","last_active":"2023-12-12T09:57:40.374324+0000","last_peered":"2023-12-12T09:57:40.374324+0000","last_clean":"2023-12-12T09:57:40.374324+0000","last_became_active":"2023-12-12T09:56:37.466591+0000","last_became_peered":"2023-12-12T09:56:37.466591+0000","last_unstale":"2023-12-12T09:57:40.374324+0000","last_undegraded":"2023-12-12T09:57:40.374324+0000","last_fullsized":"2023-12-12T09:57:40.374324+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:25:35.605587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370789+0000","last_change":"2023-12-12T09:56:38.279142+0000","last_active":"2023-12-12T09:57:28.370789+0000","last_peered":"2023-12-12T09:57:28.370789+0000","last_clean":"2023-12-12T09:57:28.370789+0000","last_became_active":"2023-12-12T09:56:38.278441+0000","last_became_peered":"2023-12-12T09:56:38.278441+0000","last_unstale":"2023-12-12T09:57:28.370789+0000","last_undegraded":"2023-12-12T09:57:28.370789+0000","last_fullsized":"2023-12-12T09:57:28.370789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:36:49.024628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370839+0000","last_change":"2023-12-12T09:56:42.506159+0000","last_active":"2023-12-12T09:57:28.370839+0000","last_peered":"2023-12-12T09:57:28.370839+0000","last_clean":"2023-12-12T09:57:28.370839+0000","last_became_active":"2023-12-12T09:56:42.505704+0000","last_became_peered":"2023-12-12T09:56:42.505704+0000","last_unstale":"2023-12-12T09:57:28.370839+0000","last_undegraded":"2023-12-12T09:57:28.370839+0000","last_fullsized":"2023-12-12T09:57:28.370839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:51:55.633237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372886+0000","last_change":"2023-12-12T09:56:37.470470+0000","last_active":"2023-12-12T09:57:28.372886+0000","last_peered":"2023-12-12T09:57:28.372886+0000","last_clean":"2023-12-12T09:57:28.372886+0000","last_became_active":"2023-12-12T09:56:37.469991+0000","last_became_peered":"2023-12-12T09:56:37.469991+0000","last_unstale":"2023-12-12T09:57:28.372886+0000","last_undegraded":"2023-12-12T09:57:28.372886+0000","last_fullsized":"2023-12-12T09:57:28.372886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:49.239075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370160+0000","last_change":"2023-12-12T09:56:38.281338+0000","last_active":"2023-12-12T09:57:28.370160+0000","last_peered":"2023-12-12T09:57:28.370160+0000","last_clean":"2023-12-12T09:57:28.370160+0000","last_became_active":"2023-12-12T09:56:38.281048+0000","last_became_peered":"2023-12-12T09:56:38.281048+0000","last_unstale":"2023-12-12T09:57:28.370160+0000","last_undegraded":"2023-12-12T09:57:28.370160+0000","last_fullsized":"2023-12-12T09:57:28.370160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:27:27.940574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371142+0000","last_change":"2023-12-12T09:56:42.500612+0000","last_active":"2023-12-12T09:57:28.371142+0000","last_peered":"2023-12-12T09:57:28.371142+0000","last_clean":"2023-12-12T09:57:28.371142+0000","last_became_active":"2023-12-12T09:56:42.500202+0000","last_became_peered":"2023-12-12T09:56:42.500202+0000","last_unstale":"2023-12-12T09:57:28.371142+0000","last_undegraded":"2023-12-12T09:57:28.371142+0000","last_fullsized":"2023-12-12T09:57:28.371142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:11:07.397950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372889+0000","last_change":"2023-12-12T09:56:37.465748+0000","last_active":"2023-12-12T09:57:28.372889+0000","last_peered":"2023-12-12T09:57:28.372889+0000","last_clean":"2023-12-12T09:57:28.372889+0000","last_became_active":"2023-12-12T09:56:37.465408+0000","last_became_peered":"2023-12-12T09:56:37.465408+0000","last_unstale":"2023-12-12T09:57:28.372889+0000","last_undegraded":"2023-12-12T09:57:28.372889+0000","last_fullsized":"2023-12-12T09:57:28.372889+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:58:38.777008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371590+0000","last_change":"2023-12-12T09:56:38.283934+0000","last_active":"2023-12-12T09:57:28.371590+0000","last_peered":"2023-12-12T09:57:28.371590+0000","last_clean":"2023-12-12T09:57:28.371590+0000","last_became_active":"2023-12-12T09:56:38.283605+0000","last_became_peered":"2023-12-12T09:56:38.283605+0000","last_unstale":"2023-12-12T09:57:28.371590+0000","last_undegraded":"2023-12-12T09:57:28.371590+0000","last_fullsized":"2023-12-12T09:57:28.371590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:52:46.792159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.364902+0000","last_change":"2023-12-12T09:56:38.279834+0000","last_active":"2023-12-12T09:57:28.364902+0000","last_peered":"2023-12-12T09:57:28.364902+0000","last_clean":"2023-12-12T09:57:28.364902+0000","last_became_active":"2023-12-12T09:56:38.279353+0000","last_became_peered":"2023-12-12T09:56:38.279353+0000","last_unstale":"2023-12-12T09:57:28.364902+0000","last_undegraded":"2023-12-12T09:57:28.364902+0000","last_fullsized":"2023-12-12T09:57:28.364902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:21:52.237458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371035+0000","last_change":"2023-12-12T09:56:38.279341+0000","last_active":"2023-12-12T09:57:28.371035+0000","last_peered":"2023-12-12T09:57:28.371035+0000","last_clean":"2023-12-12T09:57:28.371035+0000","last_became_active":"2023-12-12T09:56:38.279054+0000","last_became_peered":"2023-12-12T09:56:38.279054+0000","last_unstale":"2023-12-12T09:57:28.371035+0000","last_undegraded":"2023-12-12T09:57:28.371035+0000","last_fullsized":"2023-12-12T09:57:28.371035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:55:33.729689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370580+0000","last_change":"2023-12-12T09:56:38.277052+0000","last_active":"2023-12-12T09:57:28.370580+0000","last_peered":"2023-12-12T09:57:28.370580+0000","last_clean":"2023-12-12T09:57:28.370580+0000","last_became_active":"2023-12-12T09:56:38.276236+0000","last_became_peered":"2023-12-12T09:56:38.276236+0000","last_unstale":"2023-12-12T09:57:28.370580+0000","last_undegraded":"2023-12-12T09:57:28.370580+0000","last_fullsized":"2023-12-12T09:57:28.370580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:31:52.066130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371140+0000","last_change":"2023-12-12T09:56:38.280936+0000","last_active":"2023-12-12T09:57:28.371140+0000","last_peered":"2023-12-12T09:57:28.371140+0000","last_clean":"2023-12-12T09:57:28.371140+0000","last_became_active":"2023-12-12T09:56:38.280600+0000","last_became_peered":"2023-12-12T09:56:38.280600+0000","last_unstale":"2023-12-12T09:57:28.371140+0000","last_undegraded":"2023-12-12T09:57:28.371140+0000","last_fullsized":"2023-12-12T09:57:28.371140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:40:36.636954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371912+0000","last_change":"2023-12-12T09:56:38.278839+0000","last_active":"2023-12-12T09:57:28.371912+0000","last_peered":"2023-12-12T09:57:28.371912+0000","last_clean":"2023-12-12T09:57:28.371912+0000","last_became_active":"2023-12-12T09:56:38.276299+0000","last_became_peered":"2023-12-12T09:56:38.276299+0000","last_unstale":"2023-12-12T09:57:28.371912+0000","last_undegraded":"2023-12-12T09:57:28.371912+0000","last_fullsized":"2023-12-12T09:57:28.371912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:38:26.830037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365209+0000","last_change":"2023-12-12T09:56:38.282200+0000","last_active":"2023-12-12T09:57:28.365209+0000","last_peered":"2023-12-12T09:57:28.365209+0000","last_clean":"2023-12-12T09:57:28.365209+0000","last_became_active":"2023-12-12T09:56:38.281938+0000","last_became_peered":"2023-12-12T09:56:38.281938+0000","last_unstale":"2023-12-12T09:57:28.365209+0000","last_undegraded":"2023-12-12T09:57:28.365209+0000","last_fullsized":"2023-12-12T09:57:28.365209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:45:14.391963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371237+0000","last_change":"2023-12-12T09:56:38.276857+0000","last_active":"2023-12-12T09:57:28.371237+0000","last_peered":"2023-12-12T09:57:28.371237+0000","last_clean":"2023-12-12T09:57:28.371237+0000","last_became_active":"2023-12-12T09:56:38.276562+0000","last_became_peered":"2023-12-12T09:56:38.276562+0000","last_unstale":"2023-12-12T09:57:28.371237+0000","last_undegraded":"2023-12-12T09:57:28.371237+0000","last_fullsized":"2023-12-12T09:57:28.371237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:54:28.262613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370154+0000","last_change":"2023-12-12T09:56:38.278006+0000","last_active":"2023-12-12T09:57:28.370154+0000","last_peered":"2023-12-12T09:57:28.370154+0000","last_clean":"2023-12-12T09:57:28.370154+0000","last_became_active":"2023-12-12T09:56:38.277222+0000","last_became_peered":"2023-12-12T09:56:38.277222+0000","last_unstale":"2023-12-12T09:57:28.370154+0000","last_undegraded":"2023-12-12T09:57:28.370154+0000","last_fullsized":"2023-12-12T09:57:28.370154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:46:50.602603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371510+0000","last_change":"2023-12-12T09:56:38.282872+0000","last_active":"2023-12-12T09:57:28.371510+0000","last_peered":"2023-12-12T09:57:28.371510+0000","last_clean":"2023-12-12T09:57:28.371510+0000","last_became_active":"2023-12-12T09:56:38.282582+0000","last_became_peered":"2023-12-12T09:56:38.282582+0000","last_unstale":"2023-12-12T09:57:28.371510+0000","last_undegraded":"2023-12-12T09:57:28.371510+0000","last_fullsized":"2023-12-12T09:57:28.371510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:36:35.447323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370239+0000","last_change":"2023-12-12T09:56:38.286707+0000","last_active":"2023-12-12T09:57:28.370239+0000","last_peered":"2023-12-12T09:57:28.370239+0000","last_clean":"2023-12-12T09:57:28.370239+0000","last_became_active":"2023-12-12T09:56:38.286382+0000","last_became_peered":"2023-12-12T09:56:38.286382+0000","last_unstale":"2023-12-12T09:57:28.370239+0000","last_undegraded":"2023-12-12T09:57:28.370239+0000","last_fullsized":"2023-12-12T09:57:28.370239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:10:12.412091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371420+0000","last_change":"2023-12-12T09:56:38.276461+0000","last_active":"2023-12-12T09:57:28.371420+0000","last_peered":"2023-12-12T09:57:28.371420+0000","last_clean":"2023-12-12T09:57:28.371420+0000","last_became_active":"2023-12-12T09:56:38.276173+0000","last_became_peered":"2023-12-12T09:56:38.276173+0000","last_unstale":"2023-12-12T09:57:28.371420+0000","last_undegraded":"2023-12-12T09:57:28.371420+0000","last_fullsized":"2023-12-12T09:57:28.371420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:03:53.874132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372016+0000","last_change":"2023-12-12T09:56:38.276993+0000","last_active":"2023-12-12T09:57:28.372016+0000","last_peered":"2023-12-12T09:57:28.372016+0000","last_clean":"2023-12-12T09:57:28.372016+0000","last_became_active":"2023-12-12T09:56:38.276629+0000","last_became_peered":"2023-12-12T09:56:38.276629+0000","last_unstale":"2023-12-12T09:57:28.372016+0000","last_undegraded":"2023-12-12T09:57:28.372016+0000","last_fullsized":"2023-12-12T09:57:28.372016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:17:39.363638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365146+0000","last_change":"2023-12-12T09:56:38.281874+0000","last_active":"2023-12-12T09:57:28.365146+0000","last_peered":"2023-12-12T09:57:28.365146+0000","last_clean":"2023-12-12T09:57:28.365146+0000","last_became_active":"2023-12-12T09:56:38.281563+0000","last_became_peered":"2023-12-12T09:56:38.281563+0000","last_unstale":"2023-12-12T09:57:28.365146+0000","last_undegraded":"2023-12-12T09:57:28.365146+0000","last_fullsized":"2023-12-12T09:57:28.365146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:32:18.886026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67050,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":399,"num_read_kb":276,"num_write":239,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1144,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449112,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":491301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.48699999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.748},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.66700000000000004}]},{"osd":2,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.442},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.46700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.52900000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.63}]},{"osd":7,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.72899999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:57:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.69099999999999995}]},{"osd":9,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.58299999999999996}]},{"osd":10,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.80000000000000004}]}]},{"osd":10,"up_from":70,"seq":300647710749,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1695744,"data_stored":1137892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.57299999999999995}]},{"osd":1,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.46000000000000002},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":1.01,"15min":1.01},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.875,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.49399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.58399999999999996}]},{"osd":3,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.67600000000000005,"15min":0.69099999999999995},"min":{"1min":0.39900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97099999999999997,"5min":1.266,"15min":1.266},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.53400000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.59799999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81100000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.66800000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54900000000000004}]},{"osd":5,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82199999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.55000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.64200000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.75600000000000001,"15min":0.77100000000000002},"min":{"1min":0.51600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.996,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.61699999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0089999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.48199999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.63500000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.028,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.81399999999999995,"15min":0.82599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.71299999999999997}]},{"osd":9,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.63800000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98199999999999998,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.80500000000000005,"15min":0.82499999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.874,"5min":1.577,"15min":1.577},"last":0.79400000000000004}]},{"osd":11,"last update":"Tue Dec 12 09:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.76900000000000002}]}]},{"osd":9,"up_from":65,"seq":279172874274,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":495193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.434,"15min":0.433},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63400000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.45000000000000001,"15min":0.45600000000000002},"min":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.33100000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.64100000000000001,"15min":0.65300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.504}]},{"osd":2,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90800000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67100000000000004}]},{"osd":3,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.505,"15min":0.51300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.58299999999999996,"15min":0.59799999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86499999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.377}]},{"osd":4,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.67500000000000004,"15min":0.68899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.84299999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.752},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.75600000000000001,"15min":0.77000000000000002},"min":{"1min":0.504,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0529999999999999,"5min":1.054,"15min":1.054},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.179},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.42399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94499999999999995,"5min":1.014,"15min":1.014},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.71499999999999997}]},{"osd":8,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.79500000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Dec 12 09:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76900000000000002,"15min":0.77900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.77000000000000002}]},{"osd":11,"last update":"Tue Dec 12 09:57:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.76200000000000001}]}]},{"osd":8,"up_from":57,"seq":244813135910,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":491341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.44600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41999999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.69599999999999995}]},{"osd":2,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.307,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.497},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.44400000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.625}]},{"osd":5,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.752,"5min":0.874,"15min":0.874},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70499999999999996},"min":{"1min":0.53700000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.83199999999999996}]},{"osd":7,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.73199999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95499999999999996,"5min":1.117,"15min":1.117},"last":0.71599999999999997}]},{"osd":9,"last update":"Tue Dec 12 09:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.72399999999999998,"15min":0.71199999999999997},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76200000000000001,"15min":0.751},"min":{"1min":0.61499999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.84499999999999997}]},{"osd":10,"last update":"Tue Dec 12 09:57:36 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.76600000000000001,"15min":0.748},"min":{"1min":0.72299999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.76000000000000001,"15min":0.73799999999999999},"min":{"1min":0.77200000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.81799999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]}]},{"osd":7,"up_from":50,"seq":214748364842,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1617920,"data_stored":1087753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.311,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50900000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.435,"15min":0.41799999999999998},"min":{"1min":0.30599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60199999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59999999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57699999999999996,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.61099999999999999,"15min":0.59499999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76700000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81000000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.35899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66300000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.503,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.873,"5min":1.093,"15min":1.093},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.66300000000000003,"15min":0.64100000000000001},"min":{"1min":0.49299999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.996,"5min":1.113,"15min":1.113},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90400000000000003,"5min":1.018,"15min":1.018},"last":0.77000000000000002}]},{"osd":8,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54100000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.87}]},{"osd":9,"last update":"Tue Dec 12 09:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.60199999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78900000000000003}]},{"osd":10,"last update":"Tue Dec 12 09:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.55900000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.754},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.52300000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68200000000000005}]},{"osd":11,"last update":"Tue Dec 12 09:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.84799999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528366,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":486601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40400000000000003,"15min":0.38500000000000001},"min":{"1min":0.26200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.61499999999999999,"5min":0.624,"15min":0.624},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.42499999999999999},"min":{"1min":0.32700000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68899999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.53700000000000003},"min":{"1min":0.46999999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79600000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57799999999999996}]},{"osd":2,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.48099999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.60599999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.35199999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.73099999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.40300000000000002}]},{"osd":4,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90700000000000003,"5min":1.044,"15min":1.044},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.54800000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.90600000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.71299999999999997}]},{"osd":5,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.48899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92400000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.83099999999999996,"15min":0.86199999999999999},"min":{"1min":0.52400000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0660000000000001,"5min":5.069,"15min":5.069},"last":0.81299999999999994}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.745,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.56799999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.67300000000000004}]},{"osd":9,"last update":"Tue Dec 12 09:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.80500000000000005,"15min":0.82399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.72599999999999998}]},{"osd":10,"last update":"Tue Dec 12 09:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.82999999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":547483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53600000000000003,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.89800000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.49299999999999999},"min":{"1min":0.435,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.86399999999999999,"5min":0.87,"15min":0.87},"last":0.55500000000000005}]},{"osd":2,"last update":"Tue Dec 12 09:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.59599999999999997,"15min":0.54700000000000004},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":1.032,"15min":1.032},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91400000000000003,"5min":1.079,"15min":1.079},"last":0.70299999999999996}]},{"osd":3,"last update":"Tue Dec 12 09:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.53000000000000003},"min":{"1min":0.40100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.872,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59599999999999997,"15min":0.55000000000000004},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65200000000000002}]},{"osd":4,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.45000000000000001,"15min":0.39400000000000002},"min":{"1min":0.32300000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.503},{"interface":"front","average":{"1min":0.499,"5min":0.442,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58599999999999997,"15min":0.55500000000000005},"min":{"1min":0.40899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":1.018,"15min":1.018},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.59999999999999998,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.39200000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64100000000000001,"15min":0.60899999999999999},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57799999999999996,"15min":0.54000000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.40699999999999997,"5min":0.378,"15min":0.378},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.54100000000000004}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.48699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.69299999999999995}]},{"osd":9,"last update":"Tue Dec 12 09:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.47999999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.018,"5min":1.113,"15min":1.113},"last":0.71199999999999997}]},{"osd":10,"last update":"Tue Dec 12 09:57:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.501,"5min":0.435,"15min":0.435},"max":{"1min":0.96599999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64000000000000001}]},{"osd":11,"last update":"Tue Dec 12 09:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.72199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":488057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53800000000000003,"15min":0.49399999999999999},"min":{"1min":0.38800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.71799999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55000000000000004}]},{"osd":2,"last update":"Tue Dec 12 09:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.56399999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77400000000000002,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.55900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56599999999999995,"15min":0.53400000000000003},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48299999999999998}]},{"osd":5,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.63400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58099999999999996}]},{"osd":6,"last update":"Tue Dec 12 09:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.33000000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77400000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66400000000000003,"15min":0.67800000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96199999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.62}]},{"osd":7,"last update":"Tue Dec 12 09:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.67400000000000004}]},{"osd":9,"last update":"Tue Dec 12 09:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72099999999999997,"15min":0.72999999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98899999999999999,"5min":1.044,"15min":1.044},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.51700000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0329999999999999,"5min":1.054,"15min":1.054},"last":0.65800000000000003}]},{"osd":11,"last update":"Tue Dec 12 09:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73599999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":546750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.34999999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.79200000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.502,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.754,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.60399999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.879}]},{"osd":3,"last update":"Tue Dec 12 09:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77100000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.83499999999999996}]},{"osd":4,"last update":"Tue Dec 12 09:56:31 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.54900000000000004,"15min":0.51300000000000001},"min":{"1min":0.47099999999999997,"5min":0.378,"15min":0.378},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.56100000000000005,"15min":0.53000000000000003},"min":{"1min":0.46200000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.92500000000000004}]},{"osd":5,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45000000000000001,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.46800000000000003,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.111,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.78900000000000003}]},{"osd":7,"last update":"Tue Dec 12 09:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67600000000000005,"15min":0.67200000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.69799999999999995,"15min":0.69399999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.94499999999999995}]},{"osd":8,"last update":"Tue Dec 12 09:56:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.753}]},{"osd":9,"last update":"Tue Dec 12 09:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.89400000000000002}]},{"osd":10,"last update":"Tue Dec 12 09:57:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.77200000000000002,"15min":0.76000000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":1.04},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.78800000000000003,"15min":0.77600000000000002},"min":{"1min":0.504,"5min":0.501,"15min":0.501},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.97899999999999998}]},{"osd":11,"last update":"Tue Dec 12 09:57:36 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.99199999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":485978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.376,"15min":0.35099999999999998},"min":{"1min":0.28100000000000003,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52600000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38900000000000001,"15min":0.36299999999999999},"min":{"1min":0.31900000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56399999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.434}]},{"osd":1,"last update":"Tue Dec 12 09:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.56799999999999995,"15min":0.52600000000000002},"min":{"1min":0.45900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54100000000000004}]},{"osd":2,"last update":"Tue Dec 12 09:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90900000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58299999999999996,"15min":0.53900000000000003},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81699999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.503},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.62,"15min":0.56799999999999995},"min":{"1min":0.51500000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86599999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60399999999999998,"15min":0.58099999999999996},"min":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.434,"5min":0.432,"15min":0.432},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70999999999999996}]},{"osd":6,"last update":"Tue Dec 12 09:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.878,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.46300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.94699999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":0.70099999999999996},"min":{"1min":0.54600000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89300000000000002,"5min":1.016,"15min":1.016},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.49299999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.59699999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:55:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58099999999999996}]},{"osd":9,"last update":"Tue Dec 12 09:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.73799999999999999,"15min":0.755},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.749},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76800000000000002}]},{"osd":10,"last update":"Tue Dec 12 09:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96599999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.80200000000000005,"15min":0.81699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.93100000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.78400000000000003}]},{"osd":11,"last update":"Tue Dec 12 09:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80300000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":492073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72899999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52100000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.39900000000000002,"15min":0.38},"min":{"1min":0.32000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83999999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.443,"5min":0.373,"15min":0.373},"max":{"1min":0.82299999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48599999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83199999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40300000000000002}]},{"osd":5,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63}]},{"osd":7,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85899999999999999,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.33800000000000002}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.53800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92700000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51200000000000001}]},{"osd":9,"last update":"Tue Dec 12 09:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.502,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89600000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.53000000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56999999999999995}]},{"osd":10,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.55500000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95599999999999996,"5min":1.004,"15min":1.004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.501}]},{"osd":11,"last update":"Tue Dec 12 09:57:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61099999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790003,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1083551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.504},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51900000000000002,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.47999999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.629},{"interface":"front","average":{"1min":0.622,"5min":0.53900000000000003,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.66500000000000004}]},{"osd":2,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.27500000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.39600000000000002},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.53300000000000003}]},{"osd":3,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95199999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.89000000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.59599999999999997}]},{"osd":4,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.439,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58399999999999996,"15min":0.56299999999999994},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.628,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0029999999999999,"5min":1.056,"15min":1.056},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91100000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.49299999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.63300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97599999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997}]},{"osd":8,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.432,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.55900000000000005},"min":{"1min":0.375,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.79900000000000004}]},{"osd":9,"last update":"Tue Dec 12 09:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78800000000000003}]},{"osd":10,"last update":"Tue Dec 12 09:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.73299999999999998,"15min":0.71499999999999997},"min":{"1min":0.60999999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.752,"15min":0.73899999999999999},"min":{"1min":0.65200000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.036,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.76000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:57:48.259 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph pg dump --format=json 2023-12-12T09:57:48.895 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:49.694 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:49 smithi042 ceph-mon[98894]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:49.694 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:49 smithi042 ceph-mon[98894]: pgmap v299: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:57:49.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:49 smithi111 ceph-mon[102565]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:49.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:49 smithi111 ceph-mon[102565]: pgmap v299: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:57:49.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:49 smithi173 ceph-mon[103308]: from='client.15102 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:49.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:49 smithi173 ceph-mon[103308]: pgmap v299: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.212+0000 7f7bec9ca700 1 -- 172.21.15.42:0/3275743324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8100ef0 msgr2=0x7f7be810d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.212+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/3275743324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8100ef0 0x7f7be810d330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b0d0 tx=0x7f7be000b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.212+0000 7f7bec9ca700 1 -- 172.21.15.42:0/3275743324 shutdown_connections 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.212+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/3275743324 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7be8106560 0x7f7be8106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.212+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/3275743324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8100ef0 0x7f7be810d330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.212+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/3275743324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7be8100530 0x7f7be81009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.212+0000 7f7bec9ca700 1 -- 172.21.15.42:0/3275743324 >> 172.21.15.42:0/3275743324 conn(0x7f7be8076570 msgr2=0x7f7be8076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.212+0000 7f7bec9ca700 1 -- 172.21.15.42:0/3275743324 shutdown_connections 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.212+0000 7f7bec9ca700 1 -- 172.21.15.42:0/3275743324 wait complete. 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7bec9ca700 1 Processor -- start 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7bec9ca700 1 -- start start 2023-12-12T09:57:50.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7bec9ca700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7be8100530 0x7f7be819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7bec9ca700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7be8100ef0 0x7f7be819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7bec9ca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8106560 0x7f7be81977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7bec9ca700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be8197e50 con 0x7f7be8106560 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7bec9ca700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be8197fc0 con 0x7f7be8100ef0 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7bec9ca700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be8198180 con 0x7f7be8100530 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7be6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8106560 0x7f7be81977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7be6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8106560 0x7f7be81977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34448/0 (socket says 172.21.15.42:34448) 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7be6d9d700 1 -- 172.21.15.42:0/445233060 learned_addr learned my addr 172.21.15.42:0/445233060 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.213+0000 7f7be5d9b700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7be8100ef0 0x7f7be819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7be659c700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7be8100530 0x7f7be819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7be6d9d700 1 -- 172.21.15.42:0/445233060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7be8100530 msgr2=0x7f7be819d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7be6d9d700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7be8100530 0x7f7be819d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7be6d9d700 1 -- 172.21.15.42:0/445233060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7be8100ef0 msgr2=0x7f7be819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:50.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7be6d9d700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7be8100ef0 0x7f7be819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7be6d9d700 1 -- 172.21.15.42:0/445233060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be0009d00 con 0x7f7be8106560 2023-12-12T09:57:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7be659c700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7be8100530 0x7f7be819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7be6d9d700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8106560 0x7f7be81977e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7be0007fd0 tx=0x7f7be0009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7bd77fe700 1 -- 172.21.15.42:0/445233060 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be000e040 con 0x7f7be8106560 2023-12-12T09:57:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be8198400 con 0x7f7be8106560 2023-12-12T09:57:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7bd77fe700 1 -- 172.21.15.42:0/445233060 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7be0004c50 con 0x7f7be8106560 2023-12-12T09:57:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7bd77fe700 1 -- 172.21.15.42:0/445233060 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be001b7c0 con 0x7f7be8106560 2023-12-12T09:57:50.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.214+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be81a4690 con 0x7f7be8106560 2023-12-12T09:57:50.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.215+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be8198620 con 0x7f7be8106560 2023-12-12T09:57:50.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.216+0000 7f7bd77fe700 1 -- 172.21.15.42:0/445233060 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7be0005050 con 0x7f7be8106560 2023-12-12T09:57:50.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.217+0000 7f7bd77fe700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7bd00a1510 0x7f7bd00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:50.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.217+0000 7f7bd77fe700 1 -- 172.21.15.42:0/445233060 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7be00ce9d0 con 0x7f7be8106560 2023-12-12T09:57:50.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.217+0000 7f7be659c700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7bd00a1510 0x7f7bd00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:50.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.217+0000 7f7be659c700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7bd00a1510 0x7f7bd00a39d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc005fd0 tx=0x7f7bdc005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:50.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.221+0000 7f7bd77fe700 1 -- 172.21.15.42:0/445233060 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7be0097c50 con 0x7f7be8106560 2023-12-12T09:57:50.434 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.432+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7be8198ca0 con 0x7f7bd00a1510 2023-12-12T09:57:50.449 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.447+0000 7f7bd77fe700 1 -- 172.21.15.42:0/445233060 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427786 (secure 0 0 0) 0x7f7be8198ca0 con 0x7f7bd00a1510 2023-12-12T09:57:50.454 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:50.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7bd00a1510 msgr2=0x7f7bd00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:50.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7bd00a1510 0x7f7bd00a39d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc005fd0 tx=0x7f7bdc005e20 comp rx=0 tx=0).stop 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8106560 msgr2=0x7f7be81977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8106560 0x7f7be81977e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7be0007fd0 tx=0x7f7be0009570 comp rx=0 tx=0).stop 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 shutdown_connections 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7bd00a1510 0x7f7bd00a39d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7be8106560 0x7f7be81977e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7be8100530 0x7f7be819d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 --2- 172.21.15.42:0/445233060 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7be8100ef0 0x7f7be819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.450+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 >> 172.21.15.42:0/445233060 conn(0x7f7be8076570 msgr2=0x7f7be81044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.451+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 shutdown_connections 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:50.451+0000 7f7bec9ca700 1 -- 172.21.15.42:0/445233060 wait complete. 2023-12-12T09:57:50.459 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2023-12-12T09:57:50.856 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2023-12-12T09:57:49.826793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657664,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":575,"num_read_kb":415,"num_write":422,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181880,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":338168,"kb_used_data":13920,"kb_used_omap":18,"kb_used_meta":324077,"kb_avail":1124603656,"statfs":{"total":1151940427776,"available":1151594143744,"internally_reserved":0,"allocated":14254080,"data_stored":7833973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":331855161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003422"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366763+0000","last_change":"2023-12-12T09:56:38.273952+0000","last_active":"2023-12-12T09:57:28.366763+0000","last_peered":"2023-12-12T09:57:28.366763+0000","last_clean":"2023-12-12T09:57:28.366763+0000","last_became_active":"2023-12-12T09:56:38.273609+0000","last_became_peered":"2023-12-12T09:56:38.273609+0000","last_unstale":"2023-12-12T09:57:28.366763+0000","last_undegraded":"2023-12-12T09:57:28.366763+0000","last_fullsized":"2023-12-12T09:57:28.366763+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:23:43.144987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371202+0000","last_change":"2023-12-12T09:56:38.278719+0000","last_active":"2023-12-12T09:57:28.371202+0000","last_peered":"2023-12-12T09:57:28.371202+0000","last_clean":"2023-12-12T09:57:28.371202+0000","last_became_active":"2023-12-12T09:56:38.278404+0000","last_became_peered":"2023-12-12T09:56:38.278404+0000","last_unstale":"2023-12-12T09:57:28.371202+0000","last_undegraded":"2023-12-12T09:57:28.371202+0000","last_fullsized":"2023-12-12T09:57:28.371202+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:34:01.139106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370876+0000","last_change":"2023-12-12T09:56:38.274276+0000","last_active":"2023-12-12T09:57:28.370876+0000","last_peered":"2023-12-12T09:57:28.370876+0000","last_clean":"2023-12-12T09:57:28.370876+0000","last_became_active":"2023-12-12T09:56:38.273974+0000","last_became_peered":"2023-12-12T09:56:38.273974+0000","last_unstale":"2023-12-12T09:57:28.370876+0000","last_undegraded":"2023-12-12T09:57:28.370876+0000","last_fullsized":"2023-12-12T09:57:28.370876+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:40:55.979238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.368005+0000","last_change":"2023-12-12T09:56:38.276021+0000","last_active":"2023-12-12T09:57:28.368005+0000","last_peered":"2023-12-12T09:57:28.368005+0000","last_clean":"2023-12-12T09:57:28.368005+0000","last_became_active":"2023-12-12T09:56:38.275305+0000","last_became_peered":"2023-12-12T09:56:38.275305+0000","last_unstale":"2023-12-12T09:57:28.368005+0000","last_undegraded":"2023-12-12T09:57:28.368005+0000","last_fullsized":"2023-12-12T09:57:28.368005+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:32:28.839430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367304+0000","last_change":"2023-12-12T09:56:38.286720+0000","last_active":"2023-12-12T09:57:28.367304+0000","last_peered":"2023-12-12T09:57:28.367304+0000","last_clean":"2023-12-12T09:57:28.367304+0000","last_became_active":"2023-12-12T09:56:38.286410+0000","last_became_peered":"2023-12-12T09:56:38.286410+0000","last_unstale":"2023-12-12T09:57:28.367304+0000","last_undegraded":"2023-12-12T09:57:28.367304+0000","last_fullsized":"2023-12-12T09:57:28.367304+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:07:42.131132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365219+0000","last_change":"2023-12-12T09:56:38.273350+0000","last_active":"2023-12-12T09:57:28.365219+0000","last_peered":"2023-12-12T09:57:28.365219+0000","last_clean":"2023-12-12T09:57:28.365219+0000","last_became_active":"2023-12-12T09:56:38.273077+0000","last_became_peered":"2023-12-12T09:56:38.273077+0000","last_unstale":"2023-12-12T09:57:28.365219+0000","last_undegraded":"2023-12-12T09:57:28.365219+0000","last_fullsized":"2023-12-12T09:57:28.365219+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:27:28.081132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370577+0000","last_change":"2023-12-12T09:56:38.278212+0000","last_active":"2023-12-12T09:57:28.370577+0000","last_peered":"2023-12-12T09:57:28.370577+0000","last_clean":"2023-12-12T09:57:28.370577+0000","last_became_active":"2023-12-12T09:56:38.277913+0000","last_became_peered":"2023-12-12T09:56:38.277913+0000","last_unstale":"2023-12-12T09:57:28.370577+0000","last_undegraded":"2023-12-12T09:57:28.370577+0000","last_fullsized":"2023-12-12T09:57:28.370577+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:39:11.893381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369879+0000","last_change":"2023-12-12T09:56:38.286625+0000","last_active":"2023-12-12T09:57:28.369879+0000","last_peered":"2023-12-12T09:57:28.369879+0000","last_clean":"2023-12-12T09:57:28.369879+0000","last_became_active":"2023-12-12T09:56:38.286035+0000","last_became_peered":"2023-12-12T09:56:38.286035+0000","last_unstale":"2023-12-12T09:57:28.369879+0000","last_undegraded":"2023-12-12T09:57:28.369879+0000","last_fullsized":"2023-12-12T09:57:28.369879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:13:03.723529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372066+0000","last_change":"2023-12-12T09:56:38.281533+0000","last_active":"2023-12-12T09:57:28.372066+0000","last_peered":"2023-12-12T09:57:28.372066+0000","last_clean":"2023-12-12T09:57:28.372066+0000","last_became_active":"2023-12-12T09:56:38.281228+0000","last_became_peered":"2023-12-12T09:56:38.281228+0000","last_unstale":"2023-12-12T09:57:28.372066+0000","last_undegraded":"2023-12-12T09:57:28.372066+0000","last_fullsized":"2023-12-12T09:57:28.372066+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:49:34.075982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365595+0000","last_change":"2023-12-12T09:56:38.274599+0000","last_active":"2023-12-12T09:57:28.365595+0000","last_peered":"2023-12-12T09:57:28.365595+0000","last_clean":"2023-12-12T09:57:28.365595+0000","last_became_active":"2023-12-12T09:56:38.274309+0000","last_became_peered":"2023-12-12T09:56:38.274309+0000","last_unstale":"2023-12-12T09:57:28.365595+0000","last_undegraded":"2023-12-12T09:57:28.365595+0000","last_fullsized":"2023-12-12T09:57:28.365595+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:05:06.054345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370697+0000","last_change":"2023-12-12T09:56:38.279348+0000","last_active":"2023-12-12T09:57:28.370697+0000","last_peered":"2023-12-12T09:57:28.370697+0000","last_clean":"2023-12-12T09:57:28.370697+0000","last_became_active":"2023-12-12T09:56:38.278843+0000","last_became_peered":"2023-12-12T09:56:38.278843+0000","last_unstale":"2023-12-12T09:57:28.370697+0000","last_undegraded":"2023-12-12T09:57:28.370697+0000","last_fullsized":"2023-12-12T09:57:28.370697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:50:04.248976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367930+0000","last_change":"2023-12-12T09:56:38.276052+0000","last_active":"2023-12-12T09:57:28.367930+0000","last_peered":"2023-12-12T09:57:28.367930+0000","last_clean":"2023-12-12T09:57:28.367930+0000","last_became_active":"2023-12-12T09:56:38.275712+0000","last_became_peered":"2023-12-12T09:56:38.275712+0000","last_unstale":"2023-12-12T09:57:28.367930+0000","last_undegraded":"2023-12-12T09:57:28.367930+0000","last_fullsized":"2023-12-12T09:57:28.367930+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:37:06.573110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371935+0000","last_change":"2023-12-12T09:56:38.284052+0000","last_active":"2023-12-12T09:57:28.371935+0000","last_peered":"2023-12-12T09:57:28.371935+0000","last_clean":"2023-12-12T09:57:28.371935+0000","last_became_active":"2023-12-12T09:56:38.283741+0000","last_became_peered":"2023-12-12T09:56:38.283741+0000","last_unstale":"2023-12-12T09:57:28.371935+0000","last_undegraded":"2023-12-12T09:57:28.371935+0000","last_fullsized":"2023-12-12T09:57:28.371935+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:40:47.505972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370785+0000","last_change":"2023-12-12T09:56:38.281604+0000","last_active":"2023-12-12T09:57:28.370785+0000","last_peered":"2023-12-12T09:57:28.370785+0000","last_clean":"2023-12-12T09:57:28.370785+0000","last_became_active":"2023-12-12T09:56:38.281247+0000","last_became_peered":"2023-12-12T09:56:38.281247+0000","last_unstale":"2023-12-12T09:57:28.370785+0000","last_undegraded":"2023-12-12T09:57:28.370785+0000","last_fullsized":"2023-12-12T09:57:28.370785+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:36:14.002676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370782+0000","last_change":"2023-12-12T09:56:38.273599+0000","last_active":"2023-12-12T09:57:28.370782+0000","last_peered":"2023-12-12T09:57:28.370782+0000","last_clean":"2023-12-12T09:57:28.370782+0000","last_became_active":"2023-12-12T09:56:38.273290+0000","last_became_peered":"2023-12-12T09:56:38.273290+0000","last_unstale":"2023-12-12T09:57:28.370782+0000","last_undegraded":"2023-12-12T09:57:28.370782+0000","last_fullsized":"2023-12-12T09:57:28.370782+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:54:52.088314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370250+0000","last_change":"2023-12-12T09:56:38.275663+0000","last_active":"2023-12-12T09:57:28.370250+0000","last_peered":"2023-12-12T09:57:28.370250+0000","last_clean":"2023-12-12T09:57:28.370250+0000","last_became_active":"2023-12-12T09:56:38.275322+0000","last_became_peered":"2023-12-12T09:56:38.275322+0000","last_unstale":"2023-12-12T09:57:28.370250+0000","last_undegraded":"2023-12-12T09:57:28.370250+0000","last_fullsized":"2023-12-12T09:57:28.370250+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:46:50.067328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366661+0000","last_change":"2023-12-12T09:56:38.274064+0000","last_active":"2023-12-12T09:57:28.366661+0000","last_peered":"2023-12-12T09:57:28.366661+0000","last_clean":"2023-12-12T09:57:28.366661+0000","last_became_active":"2023-12-12T09:56:38.273733+0000","last_became_peered":"2023-12-12T09:56:38.273733+0000","last_unstale":"2023-12-12T09:57:28.366661+0000","last_undegraded":"2023-12-12T09:57:28.366661+0000","last_fullsized":"2023-12-12T09:57:28.366661+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:20:31.662432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366114+0000","last_change":"2023-12-12T09:56:38.277627+0000","last_active":"2023-12-12T09:57:28.366114+0000","last_peered":"2023-12-12T09:57:28.366114+0000","last_clean":"2023-12-12T09:57:28.366114+0000","last_became_active":"2023-12-12T09:56:38.277275+0000","last_became_peered":"2023-12-12T09:56:38.277275+0000","last_unstale":"2023-12-12T09:57:28.366114+0000","last_undegraded":"2023-12-12T09:57:28.366114+0000","last_fullsized":"2023-12-12T09:57:28.366114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:37:57.256691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370988+0000","last_change":"2023-12-12T09:56:38.276900+0000","last_active":"2023-12-12T09:57:28.370988+0000","last_peered":"2023-12-12T09:57:28.370988+0000","last_clean":"2023-12-12T09:57:28.370988+0000","last_became_active":"2023-12-12T09:56:38.276384+0000","last_became_peered":"2023-12-12T09:56:38.276384+0000","last_unstale":"2023-12-12T09:57:28.370988+0000","last_undegraded":"2023-12-12T09:57:28.370988+0000","last_fullsized":"2023-12-12T09:57:28.370988+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:13:37.097349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370598+0000","last_change":"2023-12-12T09:56:42.509239+0000","last_active":"2023-12-12T09:57:28.370598+0000","last_peered":"2023-12-12T09:57:28.370598+0000","last_clean":"2023-12-12T09:57:28.370598+0000","last_became_active":"2023-12-12T09:56:42.508699+0000","last_became_peered":"2023-12-12T09:56:42.508699+0000","last_unstale":"2023-12-12T09:57:28.370598+0000","last_undegraded":"2023-12-12T09:57:28.370598+0000","last_fullsized":"2023-12-12T09:57:28.370598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:12:39.501023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"87'13","reported_seq":71,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:43.367577+0000","last_change":"2023-12-12T09:56:37.476258+0000","last_active":"2023-12-12T09:57:43.367577+0000","last_peered":"2023-12-12T09:57:43.367577+0000","last_clean":"2023-12-12T09:57:43.367577+0000","last_became_active":"2023-12-12T09:56:37.475886+0000","last_became_peered":"2023-12-12T09:56:37.475886+0000","last_unstale":"2023-12-12T09:57:43.367577+0000","last_undegraded":"2023-12-12T09:57:43.367577+0000","last_fullsized":"2023-12-12T09:57:43.367577+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:29:34.770549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370192+0000","last_change":"2023-12-12T09:56:38.284968+0000","last_active":"2023-12-12T09:57:28.370192+0000","last_peered":"2023-12-12T09:57:28.370192+0000","last_clean":"2023-12-12T09:57:28.370192+0000","last_became_active":"2023-12-12T09:56:38.284661+0000","last_became_peered":"2023-12-12T09:56:38.284661+0000","last_unstale":"2023-12-12T09:57:28.370192+0000","last_undegraded":"2023-12-12T09:57:28.370192+0000","last_fullsized":"2023-12-12T09:57:28.370192+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:27:02.489220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.565229+0000","last_change":"2023-12-12T09:57:29.565229+0000","last_active":"2023-12-12T09:57:29.565229+0000","last_peered":"2023-12-12T09:57:29.565229+0000","last_clean":"2023-12-12T09:57:29.565229+0000","last_became_active":"2023-12-12T09:57:29.564821+0000","last_became_peered":"2023-12-12T09:57:29.564821+0000","last_unstale":"2023-12-12T09:57:29.565229+0000","last_undegraded":"2023-12-12T09:57:29.565229+0000","last_fullsized":"2023-12-12T09:57:29.565229+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:42:03.823715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370297+0000","last_change":"2023-12-12T09:56:37.471047+0000","last_active":"2023-12-12T09:57:28.370297+0000","last_peered":"2023-12-12T09:57:28.370297+0000","last_clean":"2023-12-12T09:57:28.370297+0000","last_became_active":"2023-12-12T09:56:37.470689+0000","last_became_peered":"2023-12-12T09:56:37.470689+0000","last_unstale":"2023-12-12T09:57:28.370297+0000","last_undegraded":"2023-12-12T09:57:28.370297+0000","last_fullsized":"2023-12-12T09:57:28.370297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:45:00.648165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371490+0000","last_change":"2023-12-12T09:56:38.275884+0000","last_active":"2023-12-12T09:57:28.371490+0000","last_peered":"2023-12-12T09:57:28.371490+0000","last_clean":"2023-12-12T09:57:28.371490+0000","last_became_active":"2023-12-12T09:56:38.275161+0000","last_became_peered":"2023-12-12T09:56:38.275161+0000","last_unstale":"2023-12-12T09:57:28.371490+0000","last_undegraded":"2023-12-12T09:57:28.371490+0000","last_fullsized":"2023-12-12T09:57:28.371490+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:06:02.863719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367646+0000","last_change":"2023-12-12T09:56:42.507539+0000","last_active":"2023-12-12T09:57:28.367646+0000","last_peered":"2023-12-12T09:57:28.367646+0000","last_clean":"2023-12-12T09:57:28.367646+0000","last_became_active":"2023-12-12T09:56:42.507008+0000","last_became_peered":"2023-12-12T09:56:42.507008+0000","last_unstale":"2023-12-12T09:57:28.367646+0000","last_undegraded":"2023-12-12T09:57:28.367646+0000","last_fullsized":"2023-12-12T09:57:28.367646+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:28:30.449096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"87'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367956+0000","last_change":"2023-12-12T09:56:37.475002+0000","last_active":"2023-12-12T09:57:28.367956+0000","last_peered":"2023-12-12T09:57:28.367956+0000","last_clean":"2023-12-12T09:57:28.367956+0000","last_became_active":"2023-12-12T09:56:37.474675+0000","last_became_peered":"2023-12-12T09:56:37.474675+0000","last_unstale":"2023-12-12T09:57:28.367956+0000","last_undegraded":"2023-12-12T09:57:28.367956+0000","last_fullsized":"2023-12-12T09:57:28.367956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:38:15.880299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371220+0000","last_change":"2023-12-12T09:56:38.280532+0000","last_active":"2023-12-12T09:57:28.371220+0000","last_peered":"2023-12-12T09:57:28.371220+0000","last_clean":"2023-12-12T09:57:28.371220+0000","last_became_active":"2023-12-12T09:56:38.280177+0000","last_became_peered":"2023-12-12T09:56:38.280177+0000","last_unstale":"2023-12-12T09:57:28.371220+0000","last_undegraded":"2023-12-12T09:57:28.371220+0000","last_fullsized":"2023-12-12T09:57:28.371220+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:02:34.818978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367543+0000","last_change":"2023-12-12T09:56:42.504506+0000","last_active":"2023-12-12T09:57:28.367543+0000","last_peered":"2023-12-12T09:57:28.367543+0000","last_clean":"2023-12-12T09:57:28.367543+0000","last_became_active":"2023-12-12T09:56:42.504104+0000","last_became_peered":"2023-12-12T09:56:42.504104+0000","last_unstale":"2023-12-12T09:57:28.367543+0000","last_undegraded":"2023-12-12T09:57:28.367543+0000","last_fullsized":"2023-12-12T09:57:28.367543+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:12:41.338648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367471+0000","last_change":"2023-12-12T09:56:37.475222+0000","last_active":"2023-12-12T09:57:28.367471+0000","last_peered":"2023-12-12T09:57:28.367471+0000","last_clean":"2023-12-12T09:57:28.367471+0000","last_became_active":"2023-12-12T09:56:37.474917+0000","last_became_peered":"2023-12-12T09:56:37.474917+0000","last_unstale":"2023-12-12T09:57:28.367471+0000","last_undegraded":"2023-12-12T09:57:28.367471+0000","last_fullsized":"2023-12-12T09:57:28.367471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:26:36.639215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.565521+0000","last_change":"2023-12-12T09:57:29.565521+0000","last_active":"2023-12-12T09:57:29.565521+0000","last_peered":"2023-12-12T09:57:29.565521+0000","last_clean":"2023-12-12T09:57:29.565521+0000","last_became_active":"2023-12-12T09:57:29.565139+0000","last_became_peered":"2023-12-12T09:57:29.565139+0000","last_unstale":"2023-12-12T09:57:29.565521+0000","last_undegraded":"2023-12-12T09:57:29.565521+0000","last_fullsized":"2023-12-12T09:57:29.565521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:54:53.819229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370770+0000","last_change":"2023-12-12T09:56:42.502196+0000","last_active":"2023-12-12T09:57:28.370770+0000","last_peered":"2023-12-12T09:57:28.370770+0000","last_clean":"2023-12-12T09:57:28.370770+0000","last_became_active":"2023-12-12T09:56:42.501789+0000","last_became_peered":"2023-12-12T09:56:42.501789+0000","last_unstale":"2023-12-12T09:57:28.370770+0000","last_undegraded":"2023-12-12T09:57:28.370770+0000","last_fullsized":"2023-12-12T09:57:28.370770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:04:02.656074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"87'6","reported_seq":82,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:46.402395+0000","last_change":"2023-12-12T09:57:29.386574+0000","last_active":"2023-12-12T09:57:46.402395+0000","last_peered":"2023-12-12T09:57:46.402395+0000","last_clean":"2023-12-12T09:57:46.402395+0000","last_became_active":"2023-12-12T09:57:29.378495+0000","last_became_peered":"2023-12-12T09:57:29.378495+0000","last_unstale":"2023-12-12T09:57:46.402395+0000","last_undegraded":"2023-12-12T09:57:46.402395+0000","last_fullsized":"2023-12-12T09:57:46.402395+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:47:46.745947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1010,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.368536+0000","last_change":"2023-12-12T09:56:38.281152+0000","last_active":"2023-12-12T09:57:28.368536+0000","last_peered":"2023-12-12T09:57:28.368536+0000","last_clean":"2023-12-12T09:57:28.368536+0000","last_became_active":"2023-12-12T09:56:38.280793+0000","last_became_peered":"2023-12-12T09:56:38.280793+0000","last_unstale":"2023-12-12T09:57:28.368536+0000","last_undegraded":"2023-12-12T09:57:28.368536+0000","last_fullsized":"2023-12-12T09:57:28.368536+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:06:41.297136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370821+0000","last_change":"2023-12-12T09:56:42.507439+0000","last_active":"2023-12-12T09:57:28.370821+0000","last_peered":"2023-12-12T09:57:28.370821+0000","last_clean":"2023-12-12T09:57:28.370821+0000","last_became_active":"2023-12-12T09:56:42.506980+0000","last_became_peered":"2023-12-12T09:56:42.506980+0000","last_unstale":"2023-12-12T09:57:28.370821+0000","last_undegraded":"2023-12-12T09:57:28.370821+0000","last_fullsized":"2023-12-12T09:57:28.370821+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:38:37.086804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365116+0000","last_change":"2023-12-12T09:56:37.473666+0000","last_active":"2023-12-12T09:57:28.365116+0000","last_peered":"2023-12-12T09:57:28.365116+0000","last_clean":"2023-12-12T09:57:28.365116+0000","last_became_active":"2023-12-12T09:56:37.473349+0000","last_became_peered":"2023-12-12T09:56:37.473349+0000","last_unstale":"2023-12-12T09:57:28.365116+0000","last_undegraded":"2023-12-12T09:57:28.365116+0000","last_fullsized":"2023-12-12T09:57:28.365116+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:10:06.590517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.373931+0000","last_change":"2023-12-12T09:56:38.279220+0000","last_active":"2023-12-12T09:57:28.373931+0000","last_peered":"2023-12-12T09:57:28.373931+0000","last_clean":"2023-12-12T09:57:28.373931+0000","last_became_active":"2023-12-12T09:56:38.278950+0000","last_became_peered":"2023-12-12T09:56:38.278950+0000","last_unstale":"2023-12-12T09:57:28.373931+0000","last_undegraded":"2023-12-12T09:57:28.373931+0000","last_fullsized":"2023-12-12T09:57:28.373931+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:06:24.247294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366660+0000","last_change":"2023-12-12T09:56:42.507022+0000","last_active":"2023-12-12T09:57:28.366660+0000","last_peered":"2023-12-12T09:57:28.366660+0000","last_clean":"2023-12-12T09:57:28.366660+0000","last_became_active":"2023-12-12T09:56:42.506575+0000","last_became_peered":"2023-12-12T09:56:42.506575+0000","last_unstale":"2023-12-12T09:57:28.366660+0000","last_undegraded":"2023-12-12T09:57:28.366660+0000","last_fullsized":"2023-12-12T09:57:28.366660+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:43:46.685081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371824+0000","last_change":"2023-12-12T09:56:37.464083+0000","last_active":"2023-12-12T09:57:28.371824+0000","last_peered":"2023-12-12T09:57:28.371824+0000","last_clean":"2023-12-12T09:57:28.371824+0000","last_became_active":"2023-12-12T09:56:37.463774+0000","last_became_peered":"2023-12-12T09:56:37.463774+0000","last_unstale":"2023-12-12T09:57:28.371824+0000","last_undegraded":"2023-12-12T09:57:28.371824+0000","last_fullsized":"2023-12-12T09:57:28.371824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:59:34.284076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370073+0000","last_change":"2023-12-12T09:56:38.278729+0000","last_active":"2023-12-12T09:57:28.370073+0000","last_peered":"2023-12-12T09:57:28.370073+0000","last_clean":"2023-12-12T09:57:28.370073+0000","last_became_active":"2023-12-12T09:56:38.278193+0000","last_became_peered":"2023-12-12T09:56:38.278193+0000","last_unstale":"2023-12-12T09:57:28.370073+0000","last_undegraded":"2023-12-12T09:57:28.370073+0000","last_fullsized":"2023-12-12T09:57:28.370073+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:20:25.854114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371142+0000","last_change":"2023-12-12T09:56:42.504228+0000","last_active":"2023-12-12T09:57:28.371142+0000","last_peered":"2023-12-12T09:57:28.371142+0000","last_clean":"2023-12-12T09:57:28.371142+0000","last_became_active":"2023-12-12T09:56:42.503823+0000","last_became_peered":"2023-12-12T09:56:42.503823+0000","last_unstale":"2023-12-12T09:57:28.371142+0000","last_undegraded":"2023-12-12T09:57:28.371142+0000","last_fullsized":"2023-12-12T09:57:28.371142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:54:05.263784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"87'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371368+0000","last_change":"2023-12-12T09:56:37.472108+0000","last_active":"2023-12-12T09:57:28.371368+0000","last_peered":"2023-12-12T09:57:28.371368+0000","last_clean":"2023-12-12T09:57:28.371368+0000","last_became_active":"2023-12-12T09:56:37.471822+0000","last_became_peered":"2023-12-12T09:56:37.471822+0000","last_unstale":"2023-12-12T09:57:28.371368+0000","last_undegraded":"2023-12-12T09:57:28.371368+0000","last_fullsized":"2023-12-12T09:57:28.371368+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:19:24.432739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365200+0000","last_change":"2023-12-12T09:56:38.274724+0000","last_active":"2023-12-12T09:57:28.365200+0000","last_peered":"2023-12-12T09:57:28.365200+0000","last_clean":"2023-12-12T09:57:28.365200+0000","last_became_active":"2023-12-12T09:56:38.274383+0000","last_became_peered":"2023-12-12T09:56:38.274383+0000","last_unstale":"2023-12-12T09:57:28.365200+0000","last_undegraded":"2023-12-12T09:57:28.365200+0000","last_fullsized":"2023-12-12T09:57:28.365200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:21:37.512880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370055+0000","last_change":"2023-12-12T09:56:42.505510+0000","last_active":"2023-12-12T09:57:28.370055+0000","last_peered":"2023-12-12T09:57:28.370055+0000","last_clean":"2023-12-12T09:57:28.370055+0000","last_became_active":"2023-12-12T09:56:42.504141+0000","last_became_peered":"2023-12-12T09:56:42.504141+0000","last_unstale":"2023-12-12T09:57:28.370055+0000","last_undegraded":"2023-12-12T09:57:28.370055+0000","last_fullsized":"2023-12-12T09:57:28.370055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:48:00.108207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":107,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:48.385925+0000","last_change":"2023-12-12T09:56:37.466706+0000","last_active":"2023-12-12T09:57:48.385925+0000","last_peered":"2023-12-12T09:57:48.385925+0000","last_clean":"2023-12-12T09:57:48.385925+0000","last_became_active":"2023-12-12T09:56:37.466268+0000","last_became_peered":"2023-12-12T09:56:37.466268+0000","last_unstale":"2023-12-12T09:57:48.385925+0000","last_undegraded":"2023-12-12T09:57:48.385925+0000","last_fullsized":"2023-12-12T09:57:48.385925+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:54:50.868616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371688+0000","last_change":"2023-12-12T09:56:38.277134+0000","last_active":"2023-12-12T09:57:28.371688+0000","last_peered":"2023-12-12T09:57:28.371688+0000","last_clean":"2023-12-12T09:57:28.371688+0000","last_became_active":"2023-12-12T09:56:38.276701+0000","last_became_peered":"2023-12-12T09:56:38.276701+0000","last_unstale":"2023-12-12T09:57:28.371688+0000","last_undegraded":"2023-12-12T09:57:28.371688+0000","last_fullsized":"2023-12-12T09:57:28.371688+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:46:02.519450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365168+0000","last_change":"2023-12-12T09:56:42.506494+0000","last_active":"2023-12-12T09:57:28.365168+0000","last_peered":"2023-12-12T09:57:28.365168+0000","last_clean":"2023-12-12T09:57:28.365168+0000","last_became_active":"2023-12-12T09:56:42.505965+0000","last_became_peered":"2023-12-12T09:56:42.505965+0000","last_unstale":"2023-12-12T09:57:28.365168+0000","last_undegraded":"2023-12-12T09:57:28.365168+0000","last_fullsized":"2023-12-12T09:57:28.365168+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:32:01.294474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371457+0000","last_change":"2023-12-12T09:56:37.469046+0000","last_active":"2023-12-12T09:57:28.371457+0000","last_peered":"2023-12-12T09:57:28.371457+0000","last_clean":"2023-12-12T09:57:28.371457+0000","last_became_active":"2023-12-12T09:56:37.468112+0000","last_became_peered":"2023-12-12T09:56:37.468112+0000","last_unstale":"2023-12-12T09:57:28.371457+0000","last_undegraded":"2023-12-12T09:57:28.371457+0000","last_fullsized":"2023-12-12T09:57:28.371457+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:39:35.356243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371363+0000","last_change":"2023-12-12T09:56:38.280000+0000","last_active":"2023-12-12T09:57:28.371363+0000","last_peered":"2023-12-12T09:57:28.371363+0000","last_clean":"2023-12-12T09:57:28.371363+0000","last_became_active":"2023-12-12T09:56:38.279723+0000","last_became_peered":"2023-12-12T09:56:38.279723+0000","last_unstale":"2023-12-12T09:57:28.371363+0000","last_undegraded":"2023-12-12T09:57:28.371363+0000","last_fullsized":"2023-12-12T09:57:28.371363+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:56:35.326614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367754+0000","last_change":"2023-12-12T09:56:42.506820+0000","last_active":"2023-12-12T09:57:28.367754+0000","last_peered":"2023-12-12T09:57:28.367754+0000","last_clean":"2023-12-12T09:57:28.367754+0000","last_became_active":"2023-12-12T09:56:42.506481+0000","last_became_peered":"2023-12-12T09:56:42.506481+0000","last_unstale":"2023-12-12T09:57:28.367754+0000","last_undegraded":"2023-12-12T09:57:28.367754+0000","last_fullsized":"2023-12-12T09:57:28.367754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:10:50.536967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372940+0000","last_change":"2023-12-12T09:56:37.470283+0000","last_active":"2023-12-12T09:57:28.372940+0000","last_peered":"2023-12-12T09:57:28.372940+0000","last_clean":"2023-12-12T09:57:28.372940+0000","last_became_active":"2023-12-12T09:56:37.469568+0000","last_became_peered":"2023-12-12T09:56:37.469568+0000","last_unstale":"2023-12-12T09:57:28.372940+0000","last_undegraded":"2023-12-12T09:57:28.372940+0000","last_fullsized":"2023-12-12T09:57:28.372940+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:35:24.427658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371626+0000","last_change":"2023-12-12T09:56:38.279180+0000","last_active":"2023-12-12T09:57:28.371626+0000","last_peered":"2023-12-12T09:57:28.371626+0000","last_clean":"2023-12-12T09:57:28.371626+0000","last_became_active":"2023-12-12T09:56:38.278493+0000","last_became_peered":"2023-12-12T09:56:38.278493+0000","last_unstale":"2023-12-12T09:57:28.371626+0000","last_undegraded":"2023-12-12T09:57:28.371626+0000","last_fullsized":"2023-12-12T09:57:28.371626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:17:52.468663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365640+0000","last_change":"2023-12-12T09:56:42.505140+0000","last_active":"2023-12-12T09:57:28.365640+0000","last_peered":"2023-12-12T09:57:28.365640+0000","last_clean":"2023-12-12T09:57:28.365640+0000","last_became_active":"2023-12-12T09:56:42.504655+0000","last_became_peered":"2023-12-12T09:56:42.504655+0000","last_unstale":"2023-12-12T09:57:28.365640+0000","last_undegraded":"2023-12-12T09:57:28.365640+0000","last_fullsized":"2023-12-12T09:57:28.365640+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:58:18.004084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'35","reported_seq":145,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:48.404241+0000","last_change":"2023-12-12T09:56:37.468048+0000","last_active":"2023-12-12T09:57:48.404241+0000","last_peered":"2023-12-12T09:57:48.404241+0000","last_clean":"2023-12-12T09:57:48.404241+0000","last_became_active":"2023-12-12T09:56:37.467744+0000","last_became_peered":"2023-12-12T09:56:37.467744+0000","last_unstale":"2023-12-12T09:57:48.404241+0000","last_undegraded":"2023-12-12T09:57:48.404241+0000","last_fullsized":"2023-12-12T09:57:48.404241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:29:56.033230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59852,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":91,"num_read_kb":104,"num_write":37,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367669+0000","last_change":"2023-12-12T09:56:38.284640+0000","last_active":"2023-12-12T09:57:28.367669+0000","last_peered":"2023-12-12T09:57:28.367669+0000","last_clean":"2023-12-12T09:57:28.367669+0000","last_became_active":"2023-12-12T09:56:38.284351+0000","last_became_peered":"2023-12-12T09:56:38.284351+0000","last_unstale":"2023-12-12T09:57:28.367669+0000","last_undegraded":"2023-12-12T09:57:28.367669+0000","last_fullsized":"2023-12-12T09:57:28.367669+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:16:22.910616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.373189+0000","last_change":"2023-12-12T09:56:42.502821+0000","last_active":"2023-12-12T09:57:28.373189+0000","last_peered":"2023-12-12T09:57:28.373189+0000","last_clean":"2023-12-12T09:57:28.373189+0000","last_became_active":"2023-12-12T09:56:42.502377+0000","last_became_peered":"2023-12-12T09:56:42.502377+0000","last_unstale":"2023-12-12T09:57:28.373189+0000","last_undegraded":"2023-12-12T09:57:28.373189+0000","last_fullsized":"2023-12-12T09:57:28.373189+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:41:22.913984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"87'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370931+0000","last_change":"2023-12-12T09:56:37.468656+0000","last_active":"2023-12-12T09:57:28.370931+0000","last_peered":"2023-12-12T09:57:28.370931+0000","last_clean":"2023-12-12T09:57:28.370931+0000","last_became_active":"2023-12-12T09:56:37.468353+0000","last_became_peered":"2023-12-12T09:56:37.468353+0000","last_unstale":"2023-12-12T09:57:28.370931+0000","last_undegraded":"2023-12-12T09:57:28.370931+0000","last_fullsized":"2023-12-12T09:57:28.370931+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:51:21.477423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371693+0000","last_change":"2023-12-12T09:56:38.277290+0000","last_active":"2023-12-12T09:57:28.371693+0000","last_peered":"2023-12-12T09:57:28.371693+0000","last_clean":"2023-12-12T09:57:28.371693+0000","last_became_active":"2023-12-12T09:56:38.276972+0000","last_became_peered":"2023-12-12T09:56:38.276972+0000","last_unstale":"2023-12-12T09:57:28.371693+0000","last_undegraded":"2023-12-12T09:57:28.371693+0000","last_fullsized":"2023-12-12T09:57:28.371693+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:03:25.254368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365025+0000","last_change":"2023-12-12T09:56:42.506900+0000","last_active":"2023-12-12T09:57:28.365025+0000","last_peered":"2023-12-12T09:57:28.365025+0000","last_clean":"2023-12-12T09:57:28.365025+0000","last_became_active":"2023-12-12T09:56:42.506470+0000","last_became_peered":"2023-12-12T09:56:42.506470+0000","last_unstale":"2023-12-12T09:57:28.365025+0000","last_undegraded":"2023-12-12T09:57:28.365025+0000","last_fullsized":"2023-12-12T09:57:28.365025+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:04:17.359766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"87'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371449+0000","last_change":"2023-12-12T09:56:37.472598+0000","last_active":"2023-12-12T09:57:28.371449+0000","last_peered":"2023-12-12T09:57:28.371449+0000","last_clean":"2023-12-12T09:57:28.371449+0000","last_became_active":"2023-12-12T09:56:37.471809+0000","last_became_peered":"2023-12-12T09:56:37.471809+0000","last_unstale":"2023-12-12T09:57:28.371449+0000","last_undegraded":"2023-12-12T09:57:28.371449+0000","last_fullsized":"2023-12-12T09:57:28.371449+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:54:08.677081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370962+0000","last_change":"2023-12-12T09:56:38.277180+0000","last_active":"2023-12-12T09:57:28.370962+0000","last_peered":"2023-12-12T09:57:28.370962+0000","last_clean":"2023-12-12T09:57:28.370962+0000","last_became_active":"2023-12-12T09:56:38.276878+0000","last_became_peered":"2023-12-12T09:56:38.276878+0000","last_unstale":"2023-12-12T09:57:28.370962+0000","last_undegraded":"2023-12-12T09:57:28.370962+0000","last_fullsized":"2023-12-12T09:57:28.370962+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:18:49.704820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.435169+0000","last_change":"2023-12-12T09:57:29.434750+0000","last_active":"2023-12-12T09:57:29.435169+0000","last_peered":"2023-12-12T09:57:29.435169+0000","last_clean":"2023-12-12T09:57:29.435169+0000","last_became_active":"2023-12-12T09:57:29.434302+0000","last_became_peered":"2023-12-12T09:57:29.434302+0000","last_unstale":"2023-12-12T09:57:29.435169+0000","last_undegraded":"2023-12-12T09:57:29.435169+0000","last_fullsized":"2023-12-12T09:57:29.435169+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:51:43.809887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.11","version":"87'6","reported_seq":67,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:45.359811+0000","last_change":"2023-12-12T09:56:37.468329+0000","last_active":"2023-12-12T09:57:45.359811+0000","last_peered":"2023-12-12T09:57:45.359811+0000","last_clean":"2023-12-12T09:57:45.359811+0000","last_became_active":"2023-12-12T09:56:37.467886+0000","last_became_peered":"2023-12-12T09:56:37.467886+0000","last_unstale":"2023-12-12T09:57:45.359811+0000","last_undegraded":"2023-12-12T09:57:45.359811+0000","last_fullsized":"2023-12-12T09:57:45.359811+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:23:35.796748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":976,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370135+0000","last_change":"2023-12-12T09:56:38.281280+0000","last_active":"2023-12-12T09:57:28.370135+0000","last_peered":"2023-12-12T09:57:28.370135+0000","last_clean":"2023-12-12T09:57:28.370135+0000","last_became_active":"2023-12-12T09:56:38.280993+0000","last_became_peered":"2023-12-12T09:56:38.280993+0000","last_unstale":"2023-12-12T09:57:28.370135+0000","last_undegraded":"2023-12-12T09:57:28.370135+0000","last_fullsized":"2023-12-12T09:57:28.370135+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:53:44.562981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372891+0000","last_change":"2023-12-12T09:56:42.508920+0000","last_active":"2023-12-12T09:57:28.372891+0000","last_peered":"2023-12-12T09:57:28.372891+0000","last_clean":"2023-12-12T09:57:28.372891+0000","last_became_active":"2023-12-12T09:56:42.508491+0000","last_became_peered":"2023-12-12T09:56:42.508491+0000","last_unstale":"2023-12-12T09:57:28.372891+0000","last_undegraded":"2023-12-12T09:57:28.372891+0000","last_fullsized":"2023-12-12T09:57:28.372891+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:12:50.596137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"87'17","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.395734+0000","last_change":"2023-12-12T09:57:29.395375+0000","last_active":"2023-12-12T09:57:29.395734+0000","last_peered":"2023-12-12T09:57:29.395734+0000","last_clean":"2023-12-12T09:57:29.395734+0000","last_became_active":"2023-12-12T09:57:29.380667+0000","last_became_peered":"2023-12-12T09:57:29.380667+0000","last_unstale":"2023-12-12T09:57:29.395734+0000","last_undegraded":"2023-12-12T09:57:29.395734+0000","last_fullsized":"2023-12-12T09:57:29.395734+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:16:04.815137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.381148+0000","last_change":"2023-12-12T09:57:29.381148+0000","last_active":"2023-12-12T09:57:29.381148+0000","last_peered":"2023-12-12T09:57:29.381148+0000","last_clean":"2023-12-12T09:57:29.381148+0000","last_became_active":"2023-12-12T09:57:29.380710+0000","last_became_peered":"2023-12-12T09:57:29.380710+0000","last_unstale":"2023-12-12T09:57:29.381148+0000","last_undegraded":"2023-12-12T09:57:29.381148+0000","last_fullsized":"2023-12-12T09:57:29.381148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:05:45.295141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.373394+0000","last_change":"2023-12-12T09:56:42.499828+0000","last_active":"2023-12-12T09:57:28.373394+0000","last_peered":"2023-12-12T09:57:28.373394+0000","last_clean":"2023-12-12T09:57:28.373394+0000","last_became_active":"2023-12-12T09:56:42.499453+0000","last_became_peered":"2023-12-12T09:56:42.499453+0000","last_unstale":"2023-12-12T09:57:28.373394+0000","last_undegraded":"2023-12-12T09:57:28.373394+0000","last_fullsized":"2023-12-12T09:57:28.373394+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:20:12.609145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"87'2","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.384641+0000","last_change":"2023-12-12T09:57:29.384213+0000","last_active":"2023-12-12T09:57:29.384641+0000","last_peered":"2023-12-12T09:57:29.384641+0000","last_clean":"2023-12-12T09:57:29.384641+0000","last_became_active":"2023-12-12T09:57:29.378849+0000","last_became_peered":"2023-12-12T09:57:29.378849+0000","last_unstale":"2023-12-12T09:57:29.384641+0000","last_undegraded":"2023-12-12T09:57:29.384641+0000","last_fullsized":"2023-12-12T09:57:29.384641+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:39:08.545539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365885+0000","last_change":"2023-12-12T09:56:38.281060+0000","last_active":"2023-12-12T09:57:28.365885+0000","last_peered":"2023-12-12T09:57:28.365885+0000","last_clean":"2023-12-12T09:57:28.365885+0000","last_became_active":"2023-12-12T09:56:38.280725+0000","last_became_peered":"2023-12-12T09:56:38.280725+0000","last_unstale":"2023-12-12T09:57:28.365885+0000","last_undegraded":"2023-12-12T09:57:28.365885+0000","last_fullsized":"2023-12-12T09:57:28.365885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:41:14.592213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.435128+0000","last_change":"2023-12-12T09:57:29.435128+0000","last_active":"2023-12-12T09:57:29.435128+0000","last_peered":"2023-12-12T09:57:29.435128+0000","last_clean":"2023-12-12T09:57:29.435128+0000","last_became_active":"2023-12-12T09:57:29.434622+0000","last_became_peered":"2023-12-12T09:57:29.434622+0000","last_unstale":"2023-12-12T09:57:29.435128+0000","last_undegraded":"2023-12-12T09:57:29.435128+0000","last_fullsized":"2023-12-12T09:57:29.435128+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:55.698693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"87'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365797+0000","last_change":"2023-12-12T09:56:37.465071+0000","last_active":"2023-12-12T09:57:28.365797+0000","last_peered":"2023-12-12T09:57:28.365797+0000","last_clean":"2023-12-12T09:57:28.365797+0000","last_became_active":"2023-12-12T09:56:37.464717+0000","last_became_peered":"2023-12-12T09:56:37.464717+0000","last_unstale":"2023-12-12T09:57:28.365797+0000","last_undegraded":"2023-12-12T09:57:28.365797+0000","last_fullsized":"2023-12-12T09:57:28.365797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:04:52.167228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365419+0000","last_change":"2023-12-12T09:56:38.273276+0000","last_active":"2023-12-12T09:57:28.365419+0000","last_peered":"2023-12-12T09:57:28.365419+0000","last_clean":"2023-12-12T09:57:28.365419+0000","last_became_active":"2023-12-12T09:56:38.272966+0000","last_became_peered":"2023-12-12T09:56:38.272966+0000","last_unstale":"2023-12-12T09:57:28.365419+0000","last_undegraded":"2023-12-12T09:57:28.365419+0000","last_fullsized":"2023-12-12T09:57:28.365419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:24:22.352843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371748+0000","last_change":"2023-12-12T09:56:42.510808+0000","last_active":"2023-12-12T09:57:28.371748+0000","last_peered":"2023-12-12T09:57:28.371748+0000","last_clean":"2023-12-12T09:57:28.371748+0000","last_became_active":"2023-12-12T09:56:42.510408+0000","last_became_peered":"2023-12-12T09:56:42.510408+0000","last_unstale":"2023-12-12T09:57:28.371748+0000","last_undegraded":"2023-12-12T09:57:28.371748+0000","last_fullsized":"2023-12-12T09:57:28.371748+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:53:08.573345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"87'5","reported_seq":63,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:43.368321+0000","last_change":"2023-12-12T09:56:37.468863+0000","last_active":"2023-12-12T09:57:43.368321+0000","last_peered":"2023-12-12T09:57:43.368321+0000","last_clean":"2023-12-12T09:57:43.368321+0000","last_became_active":"2023-12-12T09:56:37.468221+0000","last_became_peered":"2023-12-12T09:56:37.468221+0000","last_unstale":"2023-12-12T09:57:43.368321+0000","last_undegraded":"2023-12-12T09:57:43.368321+0000","last_fullsized":"2023-12-12T09:57:43.368321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:39:17.559717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365273+0000","last_change":"2023-12-12T09:56:38.277983+0000","last_active":"2023-12-12T09:57:28.365273+0000","last_peered":"2023-12-12T09:57:28.365273+0000","last_clean":"2023-12-12T09:57:28.365273+0000","last_became_active":"2023-12-12T09:56:38.275926+0000","last_became_peered":"2023-12-12T09:56:38.275926+0000","last_unstale":"2023-12-12T09:57:28.365273+0000","last_undegraded":"2023-12-12T09:57:28.365273+0000","last_fullsized":"2023-12-12T09:57:28.365273+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:57:51.474651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372852+0000","last_change":"2023-12-12T09:56:42.577662+0000","last_active":"2023-12-12T09:57:28.372852+0000","last_peered":"2023-12-12T09:57:28.372852+0000","last_clean":"2023-12-12T09:57:28.372852+0000","last_became_active":"2023-12-12T09:56:42.577188+0000","last_became_peered":"2023-12-12T09:56:42.577188+0000","last_unstale":"2023-12-12T09:57:28.372852+0000","last_undegraded":"2023-12-12T09:57:28.372852+0000","last_fullsized":"2023-12-12T09:57:28.372852+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:58:35.341894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371562+0000","last_change":"2023-12-12T09:56:38.279537+0000","last_active":"2023-12-12T09:57:28.371562+0000","last_peered":"2023-12-12T09:57:28.371562+0000","last_clean":"2023-12-12T09:57:28.371562+0000","last_became_active":"2023-12-12T09:56:38.278739+0000","last_became_peered":"2023-12-12T09:56:38.278739+0000","last_unstale":"2023-12-12T09:57:28.371562+0000","last_undegraded":"2023-12-12T09:57:28.371562+0000","last_fullsized":"2023-12-12T09:57:28.371562+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:35:05.056567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":67,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:47.362327+0000","last_change":"2023-12-12T09:56:37.465726+0000","last_active":"2023-12-12T09:57:47.362327+0000","last_peered":"2023-12-12T09:57:47.362327+0000","last_clean":"2023-12-12T09:57:47.362327+0000","last_became_active":"2023-12-12T09:56:37.465438+0000","last_became_peered":"2023-12-12T09:56:37.465438+0000","last_unstale":"2023-12-12T09:57:47.362327+0000","last_undegraded":"2023-12-12T09:57:47.362327+0000","last_fullsized":"2023-12-12T09:57:47.362327+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:17:58.356893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370214+0000","last_change":"2023-12-12T09:56:42.505943+0000","last_active":"2023-12-12T09:57:28.370214+0000","last_peered":"2023-12-12T09:57:28.370214+0000","last_clean":"2023-12-12T09:57:28.370214+0000","last_became_active":"2023-12-12T09:56:42.505581+0000","last_became_peered":"2023-12-12T09:56:42.505581+0000","last_unstale":"2023-12-12T09:57:28.370214+0000","last_undegraded":"2023-12-12T09:57:28.370214+0000","last_fullsized":"2023-12-12T09:57:28.370214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:19:08.869868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369564+0000","last_change":"2023-12-12T09:56:38.284895+0000","last_active":"2023-12-12T09:57:28.369564+0000","last_peered":"2023-12-12T09:57:28.369564+0000","last_clean":"2023-12-12T09:57:28.369564+0000","last_became_active":"2023-12-12T09:56:38.284429+0000","last_became_peered":"2023-12-12T09:56:38.284429+0000","last_unstale":"2023-12-12T09:57:28.369564+0000","last_undegraded":"2023-12-12T09:57:28.369564+0000","last_fullsized":"2023-12-12T09:57:28.369564+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:14.170629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"87'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367718+0000","last_change":"2023-12-12T09:56:37.466748+0000","last_active":"2023-12-12T09:57:28.367718+0000","last_peered":"2023-12-12T09:57:28.367718+0000","last_clean":"2023-12-12T09:57:28.367718+0000","last_became_active":"2023-12-12T09:56:37.466459+0000","last_became_peered":"2023-12-12T09:56:37.466459+0000","last_unstale":"2023-12-12T09:57:28.367718+0000","last_undegraded":"2023-12-12T09:57:28.367718+0000","last_fullsized":"2023-12-12T09:57:28.367718+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:21:43.819845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":324,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370077+0000","last_change":"2023-12-12T09:55:37.275186+0000","last_active":"2023-12-12T09:57:28.370077+0000","last_peered":"2023-12-12T09:57:28.370077+0000","last_clean":"2023-12-12T09:57:28.370077+0000","last_became_active":"2023-12-12T09:55:37.274808+0000","last_became_peered":"2023-12-12T09:55:37.274808+0000","last_unstale":"2023-12-12T09:57:28.370077+0000","last_undegraded":"2023-12-12T09:57:28.370077+0000","last_fullsized":"2023-12-12T09:57:28.370077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:52:42.502190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:52:42.502190+0000","last_clean_scrub_stamp":"2023-12-12T09:52:42.502190+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:27:08.942642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372950+0000","last_change":"2023-12-12T09:56:42.578219+0000","last_active":"2023-12-12T09:57:28.372950+0000","last_peered":"2023-12-12T09:57:28.372950+0000","last_clean":"2023-12-12T09:57:28.372950+0000","last_became_active":"2023-12-12T09:56:42.577589+0000","last_became_peered":"2023-12-12T09:56:42.577589+0000","last_unstale":"2023-12-12T09:57:28.372950+0000","last_undegraded":"2023-12-12T09:57:28.372950+0000","last_fullsized":"2023-12-12T09:57:28.372950+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:40:54.162235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371463+0000","last_change":"2023-12-12T09:56:38.274163+0000","last_active":"2023-12-12T09:57:28.371463+0000","last_peered":"2023-12-12T09:57:28.371463+0000","last_clean":"2023-12-12T09:57:28.371463+0000","last_became_active":"2023-12-12T09:56:38.273837+0000","last_became_peered":"2023-12-12T09:56:38.273837+0000","last_unstale":"2023-12-12T09:57:28.371463+0000","last_undegraded":"2023-12-12T09:57:28.371463+0000","last_fullsized":"2023-12-12T09:57:28.371463+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:46:53.550289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"87'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369921+0000","last_change":"2023-12-12T09:56:37.473275+0000","last_active":"2023-12-12T09:57:28.369921+0000","last_peered":"2023-12-12T09:57:28.369921+0000","last_clean":"2023-12-12T09:57:28.369921+0000","last_became_active":"2023-12-12T09:56:37.472973+0000","last_became_peered":"2023-12-12T09:56:37.472973+0000","last_unstale":"2023-12-12T09:57:28.369921+0000","last_undegraded":"2023-12-12T09:57:28.369921+0000","last_fullsized":"2023-12-12T09:57:28.369921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:18:03.660259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365011+0000","last_change":"2023-12-12T09:56:42.503299+0000","last_active":"2023-12-12T09:57:28.365011+0000","last_peered":"2023-12-12T09:57:28.365011+0000","last_clean":"2023-12-12T09:57:28.365011+0000","last_became_active":"2023-12-12T09:56:42.502862+0000","last_became_peered":"2023-12-12T09:56:42.502862+0000","last_unstale":"2023-12-12T09:57:28.365011+0000","last_undegraded":"2023-12-12T09:57:28.365011+0000","last_fullsized":"2023-12-12T09:57:28.365011+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T13:47:14.211916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"87'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367788+0000","last_change":"2023-12-12T09:56:37.476084+0000","last_active":"2023-12-12T09:57:28.367788+0000","last_peered":"2023-12-12T09:57:28.367788+0000","last_clean":"2023-12-12T09:57:28.367788+0000","last_became_active":"2023-12-12T09:56:37.475760+0000","last_became_peered":"2023-12-12T09:56:37.475760+0000","last_unstale":"2023-12-12T09:57:28.367788+0000","last_undegraded":"2023-12-12T09:57:28.367788+0000","last_fullsized":"2023-12-12T09:57:28.367788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:30:43.487809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371845+0000","last_change":"2023-12-12T09:56:38.278502+0000","last_active":"2023-12-12T09:57:28.371845+0000","last_peered":"2023-12-12T09:57:28.371845+0000","last_clean":"2023-12-12T09:57:28.371845+0000","last_became_active":"2023-12-12T09:56:38.278210+0000","last_became_peered":"2023-12-12T09:56:38.278210+0000","last_unstale":"2023-12-12T09:57:28.371845+0000","last_undegraded":"2023-12-12T09:57:28.371845+0000","last_fullsized":"2023-12-12T09:57:28.371845+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T18:25:09.539419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369978+0000","last_change":"2023-12-12T09:56:42.577598+0000","last_active":"2023-12-12T09:57:28.369978+0000","last_peered":"2023-12-12T09:57:28.369978+0000","last_clean":"2023-12-12T09:57:28.369978+0000","last_became_active":"2023-12-12T09:56:42.577147+0000","last_became_peered":"2023-12-12T09:56:42.577147+0000","last_unstale":"2023-12-12T09:57:28.369978+0000","last_undegraded":"2023-12-12T09:57:28.369978+0000","last_fullsized":"2023-12-12T09:57:28.369978+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:05:11.279871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"87'10","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366141+0000","last_change":"2023-12-12T09:56:37.474862+0000","last_active":"2023-12-12T09:57:28.366141+0000","last_peered":"2023-12-12T09:57:28.366141+0000","last_clean":"2023-12-12T09:57:28.366141+0000","last_became_active":"2023-12-12T09:56:37.474463+0000","last_became_peered":"2023-12-12T09:56:37.474463+0000","last_unstale":"2023-12-12T09:57:28.366141+0000","last_undegraded":"2023-12-12T09:57:28.366141+0000","last_fullsized":"2023-12-12T09:57:28.366141+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:26:06.926221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370049+0000","last_change":"2023-12-12T09:56:38.279729+0000","last_active":"2023-12-12T09:57:28.370049+0000","last_peered":"2023-12-12T09:57:28.370049+0000","last_clean":"2023-12-12T09:57:28.370049+0000","last_became_active":"2023-12-12T09:56:38.277634+0000","last_became_peered":"2023-12-12T09:56:38.277634+0000","last_unstale":"2023-12-12T09:57:28.370049+0000","last_undegraded":"2023-12-12T09:57:28.370049+0000","last_fullsized":"2023-12-12T09:57:28.370049+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:06:22.594507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370073+0000","last_change":"2023-12-12T09:56:42.502342+0000","last_active":"2023-12-12T09:57:28.370073+0000","last_peered":"2023-12-12T09:57:28.370073+0000","last_clean":"2023-12-12T09:57:28.370073+0000","last_became_active":"2023-12-12T09:56:42.501313+0000","last_became_peered":"2023-12-12T09:56:42.501313+0000","last_unstale":"2023-12-12T09:57:28.370073+0000","last_undegraded":"2023-12-12T09:57:28.370073+0000","last_fullsized":"2023-12-12T09:57:28.370073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:12:28.144029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"87'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.368295+0000","last_change":"2023-12-12T09:56:37.464428+0000","last_active":"2023-12-12T09:57:28.368295+0000","last_peered":"2023-12-12T09:57:28.368295+0000","last_clean":"2023-12-12T09:57:28.368295+0000","last_became_active":"2023-12-12T09:56:37.464159+0000","last_became_peered":"2023-12-12T09:56:37.464159+0000","last_unstale":"2023-12-12T09:57:28.368295+0000","last_undegraded":"2023-12-12T09:57:28.368295+0000","last_fullsized":"2023-12-12T09:57:28.368295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:14:53.779852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371245+0000","last_change":"2023-12-12T09:56:38.277023+0000","last_active":"2023-12-12T09:57:28.371245+0000","last_peered":"2023-12-12T09:57:28.371245+0000","last_clean":"2023-12-12T09:57:28.371245+0000","last_became_active":"2023-12-12T09:56:38.276708+0000","last_became_peered":"2023-12-12T09:56:38.276708+0000","last_unstale":"2023-12-12T09:57:28.371245+0000","last_undegraded":"2023-12-12T09:57:28.371245+0000","last_fullsized":"2023-12-12T09:57:28.371245+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:08:28.244491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367939+0000","last_change":"2023-12-12T09:56:42.578025+0000","last_active":"2023-12-12T09:57:28.367939+0000","last_peered":"2023-12-12T09:57:28.367939+0000","last_clean":"2023-12-12T09:57:28.367939+0000","last_became_active":"2023-12-12T09:56:42.577570+0000","last_became_peered":"2023-12-12T09:56:42.577570+0000","last_unstale":"2023-12-12T09:57:28.367939+0000","last_undegraded":"2023-12-12T09:57:28.367939+0000","last_fullsized":"2023-12-12T09:57:28.367939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:51:02.998355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"87'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370426+0000","last_change":"2023-12-12T09:56:37.467815+0000","last_active":"2023-12-12T09:57:28.370426+0000","last_peered":"2023-12-12T09:57:28.370426+0000","last_clean":"2023-12-12T09:57:28.370426+0000","last_became_active":"2023-12-12T09:56:37.467110+0000","last_became_peered":"2023-12-12T09:56:37.467110+0000","last_unstale":"2023-12-12T09:57:28.370426+0000","last_undegraded":"2023-12-12T09:57:28.370426+0000","last_fullsized":"2023-12-12T09:57:28.370426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:40:24.325774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367205+0000","last_change":"2023-12-12T09:56:38.284770+0000","last_active":"2023-12-12T09:57:28.367205+0000","last_peered":"2023-12-12T09:57:28.367205+0000","last_clean":"2023-12-12T09:57:28.367205+0000","last_became_active":"2023-12-12T09:56:38.284459+0000","last_became_peered":"2023-12-12T09:56:38.284459+0000","last_unstale":"2023-12-12T09:57:28.367205+0000","last_undegraded":"2023-12-12T09:57:28.367205+0000","last_fullsized":"2023-12-12T09:57:28.367205+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:02:42.787163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.366174+0000","last_change":"2023-12-12T09:56:42.512278+0000","last_active":"2023-12-12T09:57:28.366174+0000","last_peered":"2023-12-12T09:57:28.366174+0000","last_clean":"2023-12-12T09:57:28.366174+0000","last_became_active":"2023-12-12T09:56:42.511598+0000","last_became_peered":"2023-12-12T09:56:42.511598+0000","last_unstale":"2023-12-12T09:57:28.366174+0000","last_undegraded":"2023-12-12T09:57:28.366174+0000","last_fullsized":"2023-12-12T09:57:28.366174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:00:03.571614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"87'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370149+0000","last_change":"2023-12-12T09:56:37.469274+0000","last_active":"2023-12-12T09:57:28.370149+0000","last_peered":"2023-12-12T09:57:28.370149+0000","last_clean":"2023-12-12T09:57:28.370149+0000","last_became_active":"2023-12-12T09:56:37.468947+0000","last_became_peered":"2023-12-12T09:56:37.468947+0000","last_unstale":"2023-12-12T09:57:28.370149+0000","last_undegraded":"2023-12-12T09:57:28.370149+0000","last_fullsized":"2023-12-12T09:57:28.370149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:26:25.559385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.364832+0000","last_change":"2023-12-12T09:56:38.274415+0000","last_active":"2023-12-12T09:57:28.364832+0000","last_peered":"2023-12-12T09:57:28.364832+0000","last_clean":"2023-12-12T09:57:28.364832+0000","last_became_active":"2023-12-12T09:56:38.274119+0000","last_became_peered":"2023-12-12T09:56:38.274119+0000","last_unstale":"2023-12-12T09:57:28.364832+0000","last_undegraded":"2023-12-12T09:57:28.364832+0000","last_fullsized":"2023-12-12T09:57:28.364832+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:20:50.160355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"87'1","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367756+0000","last_change":"2023-12-12T09:56:42.506811+0000","last_active":"2023-12-12T09:57:28.367756+0000","last_peered":"2023-12-12T09:57:28.367756+0000","last_clean":"2023-12-12T09:57:28.367756+0000","last_became_active":"2023-12-12T09:56:42.506374+0000","last_became_peered":"2023-12-12T09:56:42.506374+0000","last_unstale":"2023-12-12T09:57:28.367756+0000","last_undegraded":"2023-12-12T09:57:28.367756+0000","last_fullsized":"2023-12-12T09:57:28.367756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:50:54.647403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"87'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370684+0000","last_change":"2023-12-12T09:56:37.473548+0000","last_active":"2023-12-12T09:57:28.370684+0000","last_peered":"2023-12-12T09:57:28.370684+0000","last_clean":"2023-12-12T09:57:28.370684+0000","last_became_active":"2023-12-12T09:56:37.473240+0000","last_became_peered":"2023-12-12T09:56:37.473240+0000","last_unstale":"2023-12-12T09:57:28.370684+0000","last_undegraded":"2023-12-12T09:57:28.370684+0000","last_fullsized":"2023-12-12T09:57:28.370684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:42:22.921437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.369559+0000","last_change":"2023-12-12T09:56:38.276853+0000","last_active":"2023-12-12T09:57:28.369559+0000","last_peered":"2023-12-12T09:57:28.369559+0000","last_clean":"2023-12-12T09:57:28.369559+0000","last_became_active":"2023-12-12T09:56:38.276567+0000","last_became_peered":"2023-12-12T09:56:38.276567+0000","last_unstale":"2023-12-12T09:57:28.369559+0000","last_undegraded":"2023-12-12T09:57:28.369559+0000","last_fullsized":"2023-12-12T09:57:28.369559+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:21:15.069389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.367860+0000","last_change":"2023-12-12T09:56:42.579135+0000","last_active":"2023-12-12T09:57:28.367860+0000","last_peered":"2023-12-12T09:57:28.367860+0000","last_clean":"2023-12-12T09:57:28.367860+0000","last_became_active":"2023-12-12T09:56:42.578513+0000","last_became_peered":"2023-12-12T09:56:42.578513+0000","last_unstale":"2023-12-12T09:57:28.367860+0000","last_undegraded":"2023-12-12T09:57:28.367860+0000","last_fullsized":"2023-12-12T09:57:28.367860+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:36:27.337524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"87'4","reported_seq":38,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.383906+0000","last_change":"2023-12-12T09:57:29.383906+0000","last_active":"2023-12-12T09:57:29.383906+0000","last_peered":"2023-12-12T09:57:29.383906+0000","last_clean":"2023-12-12T09:57:29.383906+0000","last_became_active":"2023-12-12T09:57:29.377499+0000","last_became_peered":"2023-12-12T09:57:29.377499+0000","last_unstale":"2023-12-12T09:57:29.383906+0000","last_undegraded":"2023-12-12T09:57:29.383906+0000","last_fullsized":"2023-12-12T09:57:29.383906+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:53:37.904144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370945+0000","last_change":"2023-12-12T09:56:38.279661+0000","last_active":"2023-12-12T09:57:28.370945+0000","last_peered":"2023-12-12T09:57:28.370945+0000","last_clean":"2023-12-12T09:57:28.370945+0000","last_became_active":"2023-12-12T09:56:38.279330+0000","last_became_peered":"2023-12-12T09:56:38.279330+0000","last_unstale":"2023-12-12T09:57:28.370945+0000","last_undegraded":"2023-12-12T09:57:28.370945+0000","last_fullsized":"2023-12-12T09:57:28.370945+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T21:19:15.049051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:29.565808+0000","last_change":"2023-12-12T09:57:29.565808+0000","last_active":"2023-12-12T09:57:29.565808+0000","last_peered":"2023-12-12T09:57:29.565808+0000","last_clean":"2023-12-12T09:57:29.565808+0000","last_became_active":"2023-12-12T09:57:29.565389+0000","last_became_peered":"2023-12-12T09:57:29.565389+0000","last_unstale":"2023-12-12T09:57:29.565808+0000","last_undegraded":"2023-12-12T09:57:29.565808+0000","last_fullsized":"2023-12-12T09:57:29.565808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T11:53:15.810984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6,7],"acting":[1,10,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"87'5","reported_seq":66,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-12-12T09:57:45.381640+0000","last_change":"2023-12-12T09:56:37.466958+0000","last_active":"2023-12-12T09:57:45.381640+0000","last_peered":"2023-12-12T09:57:45.381640+0000","last_clean":"2023-12-12T09:57:45.381640+0000","last_became_active":"2023-12-12T09:56:37.466591+0000","last_became_peered":"2023-12-12T09:56:37.466591+0000","last_unstale":"2023-12-12T09:57:45.381640+0000","last_undegraded":"2023-12-12T09:57:45.381640+0000","last_fullsized":"2023-12-12T09:57:45.381640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:25:35.605587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370789+0000","last_change":"2023-12-12T09:56:38.279142+0000","last_active":"2023-12-12T09:57:28.370789+0000","last_peered":"2023-12-12T09:57:28.370789+0000","last_clean":"2023-12-12T09:57:28.370789+0000","last_became_active":"2023-12-12T09:56:38.278441+0000","last_became_peered":"2023-12-12T09:56:38.278441+0000","last_unstale":"2023-12-12T09:57:28.370789+0000","last_undegraded":"2023-12-12T09:57:28.370789+0000","last_fullsized":"2023-12-12T09:57:28.370789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:36:49.024628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370839+0000","last_change":"2023-12-12T09:56:42.506159+0000","last_active":"2023-12-12T09:57:28.370839+0000","last_peered":"2023-12-12T09:57:28.370839+0000","last_clean":"2023-12-12T09:57:28.370839+0000","last_became_active":"2023-12-12T09:56:42.505704+0000","last_became_peered":"2023-12-12T09:56:42.505704+0000","last_unstale":"2023-12-12T09:57:28.370839+0000","last_undegraded":"2023-12-12T09:57:28.370839+0000","last_fullsized":"2023-12-12T09:57:28.370839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:51:55.633237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"87'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372886+0000","last_change":"2023-12-12T09:56:37.470470+0000","last_active":"2023-12-12T09:57:28.372886+0000","last_peered":"2023-12-12T09:57:28.372886+0000","last_clean":"2023-12-12T09:57:28.372886+0000","last_became_active":"2023-12-12T09:56:37.469991+0000","last_became_peered":"2023-12-12T09:56:37.469991+0000","last_unstale":"2023-12-12T09:57:28.372886+0000","last_undegraded":"2023-12-12T09:57:28.372886+0000","last_fullsized":"2023-12-12T09:57:28.372886+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:49:49.239075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370160+0000","last_change":"2023-12-12T09:56:38.281338+0000","last_active":"2023-12-12T09:57:28.370160+0000","last_peered":"2023-12-12T09:57:28.370160+0000","last_clean":"2023-12-12T09:57:28.370160+0000","last_became_active":"2023-12-12T09:56:38.281048+0000","last_became_peered":"2023-12-12T09:56:38.281048+0000","last_unstale":"2023-12-12T09:57:28.370160+0000","last_undegraded":"2023-12-12T09:57:28.370160+0000","last_fullsized":"2023-12-12T09:57:28.370160+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T15:27:27.940574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371142+0000","last_change":"2023-12-12T09:56:42.500612+0000","last_active":"2023-12-12T09:57:28.371142+0000","last_peered":"2023-12-12T09:57:28.371142+0000","last_clean":"2023-12-12T09:57:28.371142+0000","last_became_active":"2023-12-12T09:56:42.500202+0000","last_became_peered":"2023-12-12T09:56:42.500202+0000","last_unstale":"2023-12-12T09:57:28.371142+0000","last_undegraded":"2023-12-12T09:57:28.371142+0000","last_fullsized":"2023-12-12T09:57:28.371142+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:41.476639+0000","last_clean_scrub_stamp":"2023-12-12T09:56:41.476639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:11:07.397950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"87'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372889+0000","last_change":"2023-12-12T09:56:37.465748+0000","last_active":"2023-12-12T09:57:28.372889+0000","last_peered":"2023-12-12T09:57:28.372889+0000","last_clean":"2023-12-12T09:57:28.372889+0000","last_became_active":"2023-12-12T09:56:37.465408+0000","last_became_peered":"2023-12-12T09:56:37.465408+0000","last_unstale":"2023-12-12T09:57:28.372889+0000","last_undegraded":"2023-12-12T09:57:28.372889+0000","last_fullsized":"2023-12-12T09:57:28.372889+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:36.445619+0000","last_clean_scrub_stamp":"2023-12-12T09:56:36.445619+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:58:38.777008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371590+0000","last_change":"2023-12-12T09:56:38.283934+0000","last_active":"2023-12-12T09:57:28.371590+0000","last_peered":"2023-12-12T09:57:28.371590+0000","last_clean":"2023-12-12T09:57:28.371590+0000","last_became_active":"2023-12-12T09:56:38.283605+0000","last_became_peered":"2023-12-12T09:56:38.283605+0000","last_unstale":"2023-12-12T09:57:28.371590+0000","last_undegraded":"2023-12-12T09:57:28.371590+0000","last_fullsized":"2023-12-12T09:57:28.371590+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T12:52:46.792159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.364902+0000","last_change":"2023-12-12T09:56:38.279834+0000","last_active":"2023-12-12T09:57:28.364902+0000","last_peered":"2023-12-12T09:57:28.364902+0000","last_clean":"2023-12-12T09:57:28.364902+0000","last_became_active":"2023-12-12T09:56:38.279353+0000","last_became_peered":"2023-12-12T09:56:38.279353+0000","last_unstale":"2023-12-12T09:57:28.364902+0000","last_undegraded":"2023-12-12T09:57:28.364902+0000","last_fullsized":"2023-12-12T09:57:28.364902+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:21:52.237458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371035+0000","last_change":"2023-12-12T09:56:38.279341+0000","last_active":"2023-12-12T09:57:28.371035+0000","last_peered":"2023-12-12T09:57:28.371035+0000","last_clean":"2023-12-12T09:57:28.371035+0000","last_became_active":"2023-12-12T09:56:38.279054+0000","last_became_peered":"2023-12-12T09:56:38.279054+0000","last_unstale":"2023-12-12T09:57:28.371035+0000","last_undegraded":"2023-12-12T09:57:28.371035+0000","last_fullsized":"2023-12-12T09:57:28.371035+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:55:33.729689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370580+0000","last_change":"2023-12-12T09:56:38.277052+0000","last_active":"2023-12-12T09:57:28.370580+0000","last_peered":"2023-12-12T09:57:28.370580+0000","last_clean":"2023-12-12T09:57:28.370580+0000","last_became_active":"2023-12-12T09:56:38.276236+0000","last_became_peered":"2023-12-12T09:56:38.276236+0000","last_unstale":"2023-12-12T09:57:28.370580+0000","last_undegraded":"2023-12-12T09:57:28.370580+0000","last_fullsized":"2023-12-12T09:57:28.370580+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T14:31:52.066130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371140+0000","last_change":"2023-12-12T09:56:38.280936+0000","last_active":"2023-12-12T09:57:28.371140+0000","last_peered":"2023-12-12T09:57:28.371140+0000","last_clean":"2023-12-12T09:57:28.371140+0000","last_became_active":"2023-12-12T09:56:38.280600+0000","last_became_peered":"2023-12-12T09:56:38.280600+0000","last_unstale":"2023-12-12T09:57:28.371140+0000","last_undegraded":"2023-12-12T09:57:28.371140+0000","last_fullsized":"2023-12-12T09:57:28.371140+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T19:40:36.636954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371912+0000","last_change":"2023-12-12T09:56:38.278839+0000","last_active":"2023-12-12T09:57:28.371912+0000","last_peered":"2023-12-12T09:57:28.371912+0000","last_clean":"2023-12-12T09:57:28.371912+0000","last_became_active":"2023-12-12T09:56:38.276299+0000","last_became_peered":"2023-12-12T09:56:38.276299+0000","last_unstale":"2023-12-12T09:57:28.371912+0000","last_undegraded":"2023-12-12T09:57:28.371912+0000","last_fullsized":"2023-12-12T09:57:28.371912+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:38:26.830037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365209+0000","last_change":"2023-12-12T09:56:38.282200+0000","last_active":"2023-12-12T09:57:28.365209+0000","last_peered":"2023-12-12T09:57:28.365209+0000","last_clean":"2023-12-12T09:57:28.365209+0000","last_became_active":"2023-12-12T09:56:38.281938+0000","last_became_peered":"2023-12-12T09:56:38.281938+0000","last_unstale":"2023-12-12T09:57:28.365209+0000","last_undegraded":"2023-12-12T09:57:28.365209+0000","last_fullsized":"2023-12-12T09:57:28.365209+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:45:14.391963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371237+0000","last_change":"2023-12-12T09:56:38.276857+0000","last_active":"2023-12-12T09:57:28.371237+0000","last_peered":"2023-12-12T09:57:28.371237+0000","last_clean":"2023-12-12T09:57:28.371237+0000","last_became_active":"2023-12-12T09:56:38.276562+0000","last_became_peered":"2023-12-12T09:56:38.276562+0000","last_unstale":"2023-12-12T09:57:28.371237+0000","last_undegraded":"2023-12-12T09:57:28.371237+0000","last_fullsized":"2023-12-12T09:57:28.371237+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T16:54:28.262613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370154+0000","last_change":"2023-12-12T09:56:38.278006+0000","last_active":"2023-12-12T09:57:28.370154+0000","last_peered":"2023-12-12T09:57:28.370154+0000","last_clean":"2023-12-12T09:57:28.370154+0000","last_became_active":"2023-12-12T09:56:38.277222+0000","last_became_peered":"2023-12-12T09:56:38.277222+0000","last_unstale":"2023-12-12T09:57:28.370154+0000","last_undegraded":"2023-12-12T09:57:28.370154+0000","last_fullsized":"2023-12-12T09:57:28.370154+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:46:50.602603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371510+0000","last_change":"2023-12-12T09:56:38.282872+0000","last_active":"2023-12-12T09:57:28.371510+0000","last_peered":"2023-12-12T09:57:28.371510+0000","last_clean":"2023-12-12T09:57:28.371510+0000","last_became_active":"2023-12-12T09:56:38.282582+0000","last_became_peered":"2023-12-12T09:56:38.282582+0000","last_unstale":"2023-12-12T09:57:28.371510+0000","last_undegraded":"2023-12-12T09:57:28.371510+0000","last_fullsized":"2023-12-12T09:57:28.371510+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T20:36:35.447323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.370239+0000","last_change":"2023-12-12T09:56:38.286707+0000","last_active":"2023-12-12T09:57:28.370239+0000","last_peered":"2023-12-12T09:57:28.370239+0000","last_clean":"2023-12-12T09:57:28.370239+0000","last_became_active":"2023-12-12T09:56:38.286382+0000","last_became_peered":"2023-12-12T09:56:38.286382+0000","last_unstale":"2023-12-12T09:57:28.370239+0000","last_undegraded":"2023-12-12T09:57:28.370239+0000","last_fullsized":"2023-12-12T09:57:28.370239+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:10:12.412091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.371420+0000","last_change":"2023-12-12T09:56:38.276461+0000","last_active":"2023-12-12T09:57:28.371420+0000","last_peered":"2023-12-12T09:57:28.371420+0000","last_clean":"2023-12-12T09:57:28.371420+0000","last_became_active":"2023-12-12T09:56:38.276173+0000","last_became_peered":"2023-12-12T09:56:38.276173+0000","last_unstale":"2023-12-12T09:57:28.371420+0000","last_undegraded":"2023-12-12T09:57:28.371420+0000","last_fullsized":"2023-12-12T09:57:28.371420+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T10:03:53.874132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.372016+0000","last_change":"2023-12-12T09:56:38.276993+0000","last_active":"2023-12-12T09:57:28.372016+0000","last_peered":"2023-12-12T09:57:28.372016+0000","last_clean":"2023-12-12T09:57:28.372016+0000","last_became_active":"2023-12-12T09:56:38.276629+0000","last_became_peered":"2023-12-12T09:56:38.276629+0000","last_unstale":"2023-12-12T09:57:28.372016+0000","last_undegraded":"2023-12-12T09:57:28.372016+0000","last_fullsized":"2023-12-12T09:57:28.372016+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:17:39.363638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-12-12T09:57:28.365146+0000","last_change":"2023-12-12T09:56:38.281874+0000","last_active":"2023-12-12T09:57:28.365146+0000","last_peered":"2023-12-12T09:57:28.365146+0000","last_clean":"2023-12-12T09:57:28.365146+0000","last_became_active":"2023-12-12T09:56:38.281563+0000","last_became_peered":"2023-12-12T09:56:38.281563+0000","last_unstale":"2023-12-12T09:57:28.365146+0000","last_undegraded":"2023-12-12T09:57:28.365146+0000","last_fullsized":"2023-12-12T09:57:28.365146+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-12T09:56:37.452882+0000","last_clean_scrub_stamp":"2023-12-12T09:56:37.452882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-13T17:32:18.886026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67050,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":439,"num_read_kb":296,"num_write":239,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1144,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":78,"seq":335007449113,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":491301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45100000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.58499999999999996}]},{"osd":2,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.66400000000000003}]},{"osd":5,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.63800000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.54700000000000004}]},{"osd":7,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.70899999999999996}]},{"osd":8,"last update":"Tue Dec 12 09:57:37 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.628}]},{"osd":9,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.68500000000000005}]},{"osd":10,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.745},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.72499999999999998}]}]},{"osd":10,"up_from":70,"seq":300647710750,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28676,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716476,"statfs":{"total":95995035648,"available":95965671424,"internally_reserved":0,"allocated":1695744,"data_stored":1137892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.67800000000000005}]},{"osd":1,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.46000000000000002},"min":{"1min":0.33700000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.503},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51000000000000001,"15min":0.50800000000000001},"min":{"1min":0.307,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":1.01,"15min":1.01},"last":0.78200000000000003}]},{"osd":2,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.875,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.49399999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60799999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.67600000000000005,"15min":0.69099999999999995},"min":{"1min":0.39900000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97099999999999997,"5min":1.266,"15min":1.266},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.53400000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.59499999999999997}]},{"osd":4,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.58399999999999996,"15min":0.57799999999999996},"min":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.81100000000000005,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.54100000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.66800000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.67400000000000004,"15min":0.68200000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82199999999999995,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.751},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.55000000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93600000000000005,"5min":1,"15min":1},"last":0.69299999999999995}]},{"osd":6,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.75600000000000001,"15min":0.77100000000000002},"min":{"1min":0.51600000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.996,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.78400000000000003,"15min":0.78200000000000003},"min":{"1min":0.61699999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0089999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.82999999999999996}]},{"osd":7,"last update":"Tue Dec 12 09:56:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.48199999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.63500000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.028,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.81399999999999995,"15min":0.82599999999999996},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.84699999999999998}]},{"osd":9,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.63800000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98199999999999998,"5min":1.4370000000000001,"15min":1.4370000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.80500000000000005,"15min":0.82499999999999996},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.874,"5min":1.577,"15min":1.577},"last":0.81999999999999995}]},{"osd":11,"last update":"Tue Dec 12 09:56:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.88},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.80700000000000005}]}]},{"osd":9,"up_from":65,"seq":279172874274,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":495193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.434,"15min":0.433},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.63400000000000001,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.45000000000000001,"15min":0.45600000000000002},"min":{"1min":0.26900000000000002,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.33100000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66100000000000003,"15min":0.66600000000000004},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.64100000000000001,"15min":0.65300000000000002},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.504}]},{"osd":2,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.90800000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63500000000000001,"15min":0.64100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.67100000000000004}]},{"osd":3,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.505,"15min":0.51300000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.79400000000000004,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.58299999999999996,"15min":0.59799999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.86499999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.377}]},{"osd":4,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.67500000000000004,"15min":0.68899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.84299999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.752},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.69099999999999995}]},{"osd":5,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.65500000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.75600000000000001,"15min":0.77000000000000002},"min":{"1min":0.504,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0529999999999999,"5min":1.054,"15min":1.054},"last":0.68000000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:56:14 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":1.179},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.42399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94499999999999995,"5min":1.014,"15min":1.014},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.80400000000000005,"15min":0.81399999999999995},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.71499999999999997}]},{"osd":8,"last update":"Tue Dec 12 09:57:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79100000000000004,"15min":0.79500000000000004},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.73399999999999999}]},{"osd":10,"last update":"Tue Dec 12 09:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.76900000000000002,"15min":0.77900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.77000000000000002}]},{"osd":11,"last update":"Tue Dec 12 09:57:43 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.76200000000000001}]}]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1015808,"data_stored":491341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.44600000000000001,"5min":0.373,"15min":0.373},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.55500000000000005,"15min":0.54700000000000004},"min":{"1min":0.34300000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60299999999999998}]},{"osd":1,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.38600000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.61199999999999999,"15min":0.60799999999999998},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.626}]},{"osd":2,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.46600000000000003,"15min":0.46200000000000002},"min":{"1min":0.31900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.307,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.48399999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59399999999999997,"15min":0.59299999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.629,"15min":0.628},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68899999999999995}]},{"osd":4,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.89200000000000002,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65300000000000002,"15min":0.65500000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77400000000000002}]},{"osd":5,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.37,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.752,"5min":0.874,"15min":0.874},"last":0.78900000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.72499999999999998,"15min":0.72799999999999998},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.873},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70799999999999996,"15min":0.70499999999999996},"min":{"1min":0.53700000000000003,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.755}]},{"osd":7,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71099999999999997},"min":{"1min":0.47799999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.72899999999999998,"15min":0.73199999999999998},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.95499999999999996,"5min":1.117,"15min":1.117},"last":0.81799999999999995}]},{"osd":9,"last update":"Tue Dec 12 09:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.72399999999999998,"15min":0.71199999999999997},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76200000000000001,"15min":0.751},"min":{"1min":0.61499999999999999,"5min":0.5,"15min":0.5},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92900000000000005}]},{"osd":10,"last update":"Tue Dec 12 09:57:36 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.76600000000000001,"15min":0.748},"min":{"1min":0.72299999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.76000000000000001,"15min":0.73799999999999999},"min":{"1min":0.77200000000000002,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.95399999999999996}]},{"osd":11,"last update":"Tue Dec 12 09:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.90500000000000003}]}]},{"osd":7,"up_from":50,"seq":214748364842,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28604,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716548,"statfs":{"total":95995035648,"available":95965745152,"internally_reserved":0,"allocated":1617920,"data_stored":1087753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:32 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.52300000000000002,"15min":0.51900000000000002},"min":{"1min":0.311,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.39400000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.50900000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.435,"15min":0.41799999999999998},"min":{"1min":0.30599999999999999,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.60199999999999998,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.60899999999999999,"15min":0.60299999999999998},"min":{"1min":0.39400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.61299999999999999,"15min":0.59299999999999997},"min":{"1min":0.436,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.59999999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.57699999999999996,"15min":0.56200000000000006},"min":{"1min":0.40600000000000003,"5min":0.374,"15min":0.374},"max":{"1min":0.79600000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.61099999999999999,"15min":0.59499999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76700000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.61899999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36199999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.81000000000000005,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.35899999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.66300000000000003,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.67400000000000004,"15min":0.66500000000000004},"min":{"1min":0.503,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.873,"5min":1.093,"15min":1.093},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.66300000000000003,"15min":0.64100000000000001},"min":{"1min":0.49299999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69699999999999995,"15min":0.68899999999999995},"min":{"1min":0.47999999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.996,"5min":1.113,"15min":1.113},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67700000000000005,"15min":0.67100000000000004},"min":{"1min":0.47099999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90400000000000003,"5min":1.018,"15min":1.018},"last":0.77000000000000002}]},{"osd":8,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.71299999999999997,"15min":0.70899999999999996},"min":{"1min":0.48199999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.54100000000000004,"5min":0.498,"15min":0.498},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.87}]},{"osd":9,"last update":"Tue Dec 12 09:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.71999999999999997,"15min":0.71199999999999997},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.748,"15min":0.73999999999999999},"min":{"1min":0.60199999999999998,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.78900000000000003}]},{"osd":10,"last update":"Tue Dec 12 09:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.74299999999999999,"15min":0.73299999999999998},"min":{"1min":0.55900000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.754},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.72499999999999998,"15min":0.71499999999999997},"min":{"1min":0.52300000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68200000000000005}]},{"osd":11,"last update":"Tue Dec 12 09:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.84799999999999998}]}]},{"osd":6,"up_from":45,"seq":193273528367,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":486601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.40400000000000003,"15min":0.38500000000000001},"min":{"1min":0.26200000000000001,"5min":0.249,"15min":0.249},"max":{"1min":0.61499999999999999,"5min":0.624,"15min":0.624},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.42499999999999999},"min":{"1min":0.32700000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68899999999999995,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.33800000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59199999999999997,"15min":0.56799999999999995},"min":{"1min":0.42699999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54900000000000004,"15min":0.53700000000000003},"min":{"1min":0.46999999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79600000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.47699999999999998}]},{"osd":2,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59299999999999997,"15min":0.57499999999999996},"min":{"1min":0.48099999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.51000000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.548,"5min":1.548,"15min":1.548},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.5,"15min":0.47899999999999998},"min":{"1min":0.35199999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.73099999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.5},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.314,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.39100000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.623,"15min":0.60099999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90700000000000003,"5min":1.044,"15min":1.044},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.65900000000000003,"15min":0.64100000000000001},"min":{"1min":0.54800000000000004,"5min":0.37,"15min":0.37},"max":{"1min":0.90600000000000003,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.60499999999999998}]},{"osd":5,"last update":"Tue Dec 12 09:57:14 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.66700000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.73599999999999999}]},{"osd":7,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68500000000000005,"15min":0.66700000000000004},"min":{"1min":0.48899999999999999,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.92400000000000004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.83099999999999996,"15min":0.86199999999999999},"min":{"1min":0.52400000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0660000000000001,"5min":5.069,"15min":5.069},"last":0.621}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.745,"15min":0.73999999999999999},"min":{"1min":0.53500000000000003,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.73899999999999999,"15min":0.73099999999999998},"min":{"1min":0.56799999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.78800000000000003}]},{"osd":9,"last update":"Tue Dec 12 09:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.80500000000000005,"15min":0.82399999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.53800000000000003}]},{"osd":10,"last update":"Tue Dec 12 09:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.80200000000000005,"15min":0.79900000000000004},"min":{"1min":0.64300000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.86399999999999999,"15min":0.86899999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.65600000000000003}]},{"osd":11,"last update":"Tue Dec 12 09:57:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.1760000000000002,"5min":2.1760000000000002,"15min":2.1760000000000002},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.88200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574917,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":547483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.53600000000000003,"15min":0.502},"min":{"1min":0.35399999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.89800000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53400000000000003,"15min":0.49299999999999999},"min":{"1min":0.435,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.86399999999999999,"5min":0.87,"15min":0.87},"last":0.55500000000000005}]},{"osd":2,"last update":"Tue Dec 12 09:56:48 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.59599999999999997,"15min":0.54700000000000004},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.96299999999999997,"5min":1.032,"15min":1.032},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.57699999999999996,"15min":0.54600000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91400000000000003,"5min":1.079,"15min":1.079},"last":0.70299999999999996}]},{"osd":3,"last update":"Tue Dec 12 09:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.53000000000000003},"min":{"1min":0.40100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.872,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.63},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59599999999999997,"15min":0.55000000000000004},"min":{"1min":0.41799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84899999999999998,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.65200000000000002}]},{"osd":4,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.45000000000000001,"15min":0.39400000000000002},"min":{"1min":0.32300000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.503},{"interface":"front","average":{"1min":0.499,"5min":0.442,"15min":0.40699999999999997},"min":{"1min":0.30299999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.51700000000000002}]},{"osd":5,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.58599999999999997,"15min":0.55500000000000005},"min":{"1min":0.40899999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.86399999999999999,"5min":1.018,"15min":1.018},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.59999999999999998,"15min":0.57299999999999995},"min":{"1min":0.42199999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":0.66400000000000003}]},{"osd":6,"last update":"Tue Dec 12 09:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.59599999999999997,"15min":0.57699999999999996},"min":{"1min":0.39200000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.64100000000000001,"15min":0.60899999999999999},"min":{"1min":0.46999999999999997,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.68100000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.57799999999999996,"15min":0.54000000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.61199999999999999,"15min":0.57799999999999996},"min":{"1min":0.40699999999999997,"5min":0.378,"15min":0.378},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.54100000000000004}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.69499999999999995,"15min":0.67700000000000005},"min":{"1min":0.48699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.70699999999999996,"15min":0.69699999999999995},"min":{"1min":0.48299999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.69299999999999995}]},{"osd":9,"last update":"Tue Dec 12 09:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73399999999999999,"15min":0.73199999999999998},"min":{"1min":0.47999999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55200000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.018,"5min":1.113,"15min":1.113},"last":0.71199999999999997}]},{"osd":10,"last update":"Tue Dec 12 09:57:36 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.76500000000000001,"15min":0.76000000000000001},"min":{"1min":0.501,"5min":0.435,"15min":0.435},"max":{"1min":0.96599999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.52500000000000002,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64000000000000001}]},{"osd":11,"last update":"Tue Dec 12 09:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.72199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":488057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 12 09:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.53800000000000003,"15min":0.49399999999999999},"min":{"1min":0.38800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.71799999999999997,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.56000000000000005,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.79100000000000004,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.55000000000000004}]},{"osd":2,"last update":"Tue Dec 12 09:56:51 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61099999999999999,"15min":0.56399999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.77400000000000002,"5min":4.6630000000000003,"15min":4.6630000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.59999999999999998,"15min":0.55900000000000005},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Dec 12 09:57:39 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Dec 12 09:57:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.56599999999999995,"15min":0.53400000000000003},"min":{"1min":0.38500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.61699999999999999,"15min":0.58899999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.48299999999999998}]},{"osd":5,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.59799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.64100000000000001,"15min":0.63400000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.58099999999999996}]},{"osd":6,"last update":"Tue Dec 12 09:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.33000000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.77400000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.66400000000000003,"15min":0.67800000000000005},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.96199999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.62}]},{"osd":7,"last update":"Tue Dec 12 09:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.52400000000000002}]},{"osd":8,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.68100000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.76800000000000002,"15min":0.77800000000000002},"min":{"1min":0.53400000000000003,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.67400000000000004}]},{"osd":9,"last update":"Tue Dec 12 09:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.72099999999999997,"15min":0.72999999999999998},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.98899999999999999,"5min":1.044,"15min":1.044},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.64900000000000002}]},{"osd":10,"last update":"Tue Dec 12 09:57:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.51700000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.77200000000000002,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0329999999999999,"5min":1.054,"15min":1.054},"last":0.65800000000000003}]},{"osd":11,"last update":"Tue Dec 12 09:57:00 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.73599999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":546750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51300000000000001,"15min":0.50600000000000001},"min":{"1min":0.34999999999999998,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.79200000000000004,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.502,"15min":0.497},"min":{"1min":0.33300000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.754,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.43099999999999999}]},{"osd":1,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.50900000000000001},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.53800000000000003,"15min":0.51500000000000001},"min":{"1min":0.33200000000000002,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.75600000000000001}]},{"osd":3,"last update":"Tue Dec 12 09:57:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.40200000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.77100000000000002,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.51800000000000002}]},{"osd":4,"last update":"Tue Dec 12 09:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.74299999999999999}]},{"osd":5,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.45000000000000001,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.46800000000000003,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Dec 12 09:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.755,"15min":0.79500000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.111,"5min":4.8869999999999996,"15min":4.8869999999999996},"last":0.65900000000000003}]},{"osd":7,"last update":"Tue Dec 12 09:57:42 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.79000000000000004}]},{"osd":8,"last update":"Tue Dec 12 09:56:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.60299999999999998,"15min":0.59399999999999997},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.64100000000000001,"15min":0.63300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.626}]},{"osd":9,"last update":"Tue Dec 12 09:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.752,"15min":0.75600000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.67900000000000005}]},{"osd":10,"last update":"Tue Dec 12 09:57:36 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.77200000000000002,"15min":0.76000000000000001},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.78800000000000003,"15min":0.77600000000000002},"min":{"1min":0.504,"5min":0.501,"15min":0.501},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.83799999999999997}]},{"osd":11,"last update":"Tue Dec 12 09:57:36 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.72299999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182459,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":485978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.376,"15min":0.35099999999999998},"min":{"1min":0.28100000000000003,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.52600000000000002,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.38900000000000001,"15min":0.36299999999999999},"min":{"1min":0.31900000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.56399999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.434}]},{"osd":1,"last update":"Tue Dec 12 09:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.56799999999999995,"15min":0.52600000000000002},"min":{"1min":0.45900000000000002,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56799999999999995,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.54100000000000004}]},{"osd":2,"last update":"Tue Dec 12 09:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57799999999999996,"15min":0.56499999999999995},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59899999999999998,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90900000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.72899999999999998}]},{"osd":4,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58299999999999996,"15min":0.53900000000000003},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81699999999999995,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.503},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.62,"15min":0.56799999999999995},"min":{"1min":0.51500000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.86599999999999999,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60399999999999998,"15min":0.58099999999999996},"min":{"1min":0.41999999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.434,"5min":0.432,"15min":0.432},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.70999999999999996}]},{"osd":6,"last update":"Tue Dec 12 09:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53500000000000003,"15min":0.52700000000000002},"min":{"1min":0.378,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.878,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.60399999999999998,"15min":0.56999999999999995},"min":{"1min":0.46300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.94699999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.71199999999999997,"15min":0.70099999999999996},"min":{"1min":0.54600000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89300000000000002,"5min":1.016,"15min":1.016},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.69299999999999995,"15min":0.68300000000000005},"min":{"1min":0.49299999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.59699999999999998}]},{"osd":8,"last update":"Tue Dec 12 09:55:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.58099999999999996}]},{"osd":9,"last update":"Tue Dec 12 09:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.73799999999999999,"15min":0.755},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.749},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.77300000000000002,"15min":0.77900000000000003},"min":{"1min":0.55300000000000005,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76800000000000002}]},{"osd":10,"last update":"Tue Dec 12 09:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.80800000000000005,"15min":0.81999999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":0.96599999999999997,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.80200000000000005,"15min":0.81699999999999995},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":0.93100000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.78400000000000003}]},{"osd":11,"last update":"Tue Dec 12 09:57:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.80300000000000005}]}]},{"osd":4,"up_from":32,"seq":137438953528,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":492073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:33 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.72899999999999998,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.29999999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52100000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.39900000000000002,"15min":0.38},"min":{"1min":0.32000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.54800000000000004,"15min":0.54900000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.83999999999999997,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.55900000000000005,"15min":0.53000000000000003},"min":{"1min":0.443,"5min":0.373,"15min":0.373},"max":{"1min":0.82299999999999995,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.48599999999999999}]},{"osd":3,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.59799999999999998,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.59199999999999997,"15min":0.59799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83199999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40300000000000002}]},{"osd":5,"last update":"Tue Dec 12 09:56:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.58799999999999997,"15min":0.57799999999999996},"min":{"1min":0.46300000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.55600000000000005}]},{"osd":6,"last update":"Tue Dec 12 09:57:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.42099999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.84799999999999998,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.67100000000000004,"15min":0.66700000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.63}]},{"osd":7,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.56000000000000005,"15min":0.53800000000000003},"min":{"1min":0.40000000000000002,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85899999999999999,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57499999999999996,"15min":0.55900000000000005},"min":{"1min":0.38200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.33800000000000002}]},{"osd":8,"last update":"Tue Dec 12 09:56:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.53800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.92700000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42399999999999999,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.51200000000000001}]},{"osd":9,"last update":"Tue Dec 12 09:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.71399999999999997,"15min":0.71599999999999997},"min":{"1min":0.502,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":0.89600000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69899999999999995,"15min":0.69699999999999995},"min":{"1min":0.53000000000000003,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.56999999999999995}]},{"osd":10,"last update":"Tue Dec 12 09:57:34 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.74099999999999999,"15min":0.73899999999999999},"min":{"1min":0.55500000000000005,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.95599999999999996,"5min":1.004,"15min":1.004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.67300000000000004,"15min":0.67500000000000004},"min":{"1min":0.42599999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.501}]},{"osd":11,"last update":"Tue Dec 12 09:57:37 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.61099999999999999}]}]},{"osd":5,"up_from":37,"seq":158913790004,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1083551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52000000000000002,"15min":0.51200000000000001},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.51900000000000002,"15min":0.51000000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52200000000000002}]},{"osd":1,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.55100000000000005,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.53900000000000003,"15min":0.51800000000000002},"min":{"1min":0.376,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.67300000000000004}]},{"osd":2,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.40200000000000002,"15min":0.38},"min":{"1min":0.27500000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.41199999999999998,"15min":0.39600000000000002},"min":{"1min":0.29099999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.33300000000000002}]},{"osd":3,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.58899999999999997,"15min":0.57799999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.95199999999999996,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.439},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57899999999999996,"15min":0.57099999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.89000000000000001,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Tue Dec 12 09:56:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.58899999999999997,"15min":0.57099999999999995},"min":{"1min":0.439,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.58399999999999996,"15min":0.56299999999999994},"min":{"1min":0.40600000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Dec 12 09:57:11 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.628,"15min":0.60399999999999998},"min":{"1min":0.39200000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0029999999999999,"5min":1.056,"15min":1.056},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.63600000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91100000000000003,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Dec 12 09:57:32 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.67000000000000004,"15min":0.65200000000000002},"min":{"1min":0.49299999999999999,"5min":0.441,"15min":0.441},"max":{"1min":0.98799999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.71199999999999997,"15min":0.69699999999999995},"min":{"1min":0.63300000000000001,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.97599999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.74399999999999999}]},{"osd":8,"last update":"Tue Dec 12 09:56:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.55600000000000005,"15min":0.54100000000000004},"min":{"1min":0.432,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.57399999999999995,"15min":0.55900000000000005},"min":{"1min":0.375,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.47899999999999998}]},{"osd":9,"last update":"Tue Dec 12 09:57:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.69899999999999995},"min":{"1min":0.41799999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73399999999999999},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.629}]},{"osd":10,"last update":"Tue Dec 12 09:57:38 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.73299999999999998,"15min":0.71499999999999997},"min":{"1min":0.60999999999999999,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.752,"15min":0.73899999999999999},"min":{"1min":0.65200000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.036,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.66100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-12T09:57:50.863 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-12-12T09:57:50.863 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-12T09:57:50.864 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-12-12T09:57:50.864 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph health --format=json 2023-12-12T09:57:51.465 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:57:51.697 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:51 smithi042 ceph-mon[98894]: pgmap v300: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:57:51.697 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:51 smithi042 ceph-mon[98894]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:51.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:51 smithi111 ceph-mon[102565]: pgmap v300: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:57:51.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:51 smithi111 ceph-mon[102565]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:51 smithi173 ceph-mon[103308]: pgmap v300: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:57:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:51 smithi173 ceph-mon[103308]: from='client.15108 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-12T09:57:52.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.768+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3925505897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100ef0 msgr2=0x7fbb4810d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:52.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.768+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3925505897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100ef0 0x7fbb4810d330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000b0d0 tx=0x7fbb4000b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:52.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.768+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3925505897 shutdown_connections 2023-12-12T09:57:52.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.768+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3925505897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100ef0 0x7fbb4810d330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:52.771 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.768+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3925505897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb48106560 0x7fbb48106940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:52.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.768+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3925505897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb48100530 0x7fbb481009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:52.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.768+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3925505897 >> 172.21.15.42:0/3925505897 conn(0x7fbb48076570 msgr2=0x7fbb48076980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:52.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.768+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3925505897 shutdown_connections 2023-12-12T09:57:52.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.768+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3925505897 wait complete. 2023-12-12T09:57:52.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb4e75b700 1 Processor -- start 2023-12-12T09:57:52.772 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb4e75b700 1 -- start start 2023-12-12T09:57:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb4e75b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100530 0x7fbb48102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb4e75b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb48100ef0 0x7fbb48102660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb4e75b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb48106560 0x7fbb48102ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb4e75b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481a5de0 con 0x7fbb48100530 2023-12-12T09:57:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb4e75b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481a5f50 con 0x7fbb48100ef0 2023-12-12T09:57:52.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb4e75b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb481a6110 con 0x7fbb48106560 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb47fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100530 0x7fbb48102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb47fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100530 0x7fbb48102120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34456/0 (socket says 172.21.15.42:34456) 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb47fff700 1 -- 172.21.15.42:0/3800035407 learned_addr learned my addr 172.21.15.42:0/3800035407 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.769+0000 7fbb477fe700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb48100ef0 0x7fbb48102660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb47fff700 1 -- 172.21.15.42:0/3800035407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb48106560 msgr2=0x7fbb48102ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb47fff700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb48106560 0x7fbb48102ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb47fff700 1 -- 172.21.15.42:0/3800035407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb48100ef0 msgr2=0x7fbb48102660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb47fff700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb48100ef0 0x7fbb48102660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb47fff700 1 -- 172.21.15.42:0/3800035407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40009d00 con 0x7fbb48100530 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb477fe700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb48100ef0 0x7fbb48102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb47fff700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100530 0x7fbb48102120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800d950 tx=0x7fbb3800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:52.774 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb457fa700 1 -- 172.21.15.42:0/3800035407 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38009b40 con 0x7fbb48100530 2023-12-12T09:57:52.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb481a63f0 con 0x7fbb48100530 2023-12-12T09:57:52.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb457fa700 1 -- 172.21.15.42:0/3800035407 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb38009ca0 con 0x7fbb48100530 2023-12-12T09:57:52.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb457fa700 1 -- 172.21.15.42:0/3800035407 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3800f620 con 0x7fbb48100530 2023-12-12T09:57:52.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.770+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb481a68f0 con 0x7fbb48100530 2023-12-12T09:57:52.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.771+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb48101bd0 con 0x7fbb48100530 2023-12-12T09:57:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.773+0000 7fbb457fa700 1 -- 172.21.15.42:0/3800035407 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fbb38010460 con 0x7fbb48100530 2023-12-12T09:57:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.773+0000 7fbb457fa700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb300a1510 0x7fbb300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.773+0000 7fbb477fe700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb300a1510 0x7fbb300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.773+0000 7fbb457fa700 1 -- 172.21.15.42:0/3800035407 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbb380cd530 con 0x7fbb48100530 2023-12-12T09:57:52.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.774+0000 7fbb477fe700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb300a1510 0x7fbb300a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005d90 tx=0x7fbb3c005d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:52.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:52.777+0000 7fbb457fa700 1 -- 172.21.15.42:0/3800035407 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbb380966c0 con 0x7fbb48100530 2023-12-12T09:57:53.021 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.018+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbb4804f290 con 0x7fbb48100530 2023-12-12T09:57:53.021 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.019+0000 7fbb457fa700 1 -- 172.21.15.42:0/3800035407 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fbb4804f290 con 0x7fbb48100530 2023-12-12T09:57:53.021 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:53.021 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb300a1510 msgr2=0x7fbb300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb300a1510 0x7fbb300a39d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c005d90 tx=0x7fbb3c005d20 comp rx=0 tx=0).stop 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100530 msgr2=0x7fbb48102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100530 0x7fbb48102120 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800d950 tx=0x7fbb3800dd10 comp rx=0 tx=0).stop 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 shutdown_connections 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fbb300a1510 0x7fbb300a39d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb48106560 0x7fbb48102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbb48100530 0x7fbb48102120 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 --2- 172.21.15.42:0/3800035407 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbb48100ef0 0x7fbb48102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 >> 172.21.15.42:0/3800035407 conn(0x7fbb48076570 msgr2=0x7fbb480ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:53.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 shutdown_connections 2023-12-12T09:57:53.024 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.021+0000 7fbb4e75b700 1 -- 172.21.15.42:0/3800035407 wait complete. 2023-12-12T09:57:53.428 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-12-12T09:57:53.428 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:57:53.560 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.556+0000 7face846f700 1 -- 172.21.15.42:0/110294387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face01027e0 msgr2=0x7face0102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.556+0000 7face846f700 1 --2- 172.21.15.42:0/110294387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face01027e0 0x7face0102c40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7facdc009990 tx=0x7facdc009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.556+0000 7face846f700 1 -- 172.21.15.42:0/110294387 shutdown_connections 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.556+0000 7face846f700 1 --2- 172.21.15.42:0/110294387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face00ff770 0x7face00ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.556+0000 7face846f700 1 --2- 172.21.15.42:0/110294387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7face0103180 0x7face0109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.556+0000 7face846f700 1 --2- 172.21.15.42:0/110294387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face01027e0 0x7face0102c40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.556+0000 7face846f700 1 -- 172.21.15.42:0/110294387 >> 172.21.15.42:0/110294387 conn(0x7face00fb0a0 msgr2=0x7face00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.556+0000 7face846f700 1 -- 172.21.15.42:0/110294387 shutdown_connections 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.556+0000 7face846f700 1 -- 172.21.15.42:0/110294387 wait complete. 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face846f700 1 Processor -- start 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face846f700 1 -- start start 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face846f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face00ff770 0x7face0197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face846f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face01027e0 0x7face01979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:53.561 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face846f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7face0103180 0x7face019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face846f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face019ef90 con 0x7face00ff770 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face846f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face019f0d0 con 0x7face01027e0 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face846f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face019f260 con 0x7face0103180 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face620b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face00ff770 0x7face0197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face620b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face00ff770 0x7face0197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34468/0 (socket says 172.21.15.42:34468) 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face620b700 1 -- 172.21.15.42:0/2677835182 learned_addr learned my addr 172.21.15.42:0/2677835182 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.557+0000 7face5a0a700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face01027e0 0x7face01979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face6a0c700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7face0103180 0x7face019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face620b700 1 -- 172.21.15.42:0/2677835182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7face0103180 msgr2=0x7face019e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face620b700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7face0103180 0x7face019e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face620b700 1 -- 172.21.15.42:0/2677835182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face01027e0 msgr2=0x7face01979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face620b700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face01027e0 0x7face01979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.562 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face620b700 1 -- 172.21.15.42:0/2677835182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc0095f0 con 0x7face00ff770 2023-12-12T09:57:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face6a0c700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7face0103180 0x7face019e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face5a0a700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face01027e0 0x7face01979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face620b700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face00ff770 0x7face0197460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7facd000c970 tx=0x7facd000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7facd77fe700 1 -- 172.21.15.42:0/2677835182 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0004220 con 0x7face00ff770 2023-12-12T09:57:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face019f540 con 0x7face00ff770 2023-12-12T09:57:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7facd77fe700 1 -- 172.21.15.42:0/2677835182 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facd0007d60 con 0x7face00ff770 2023-12-12T09:57:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7facd77fe700 1 -- 172.21.15.42:0/2677835182 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0003c90 con 0x7face00ff770 2023-12-12T09:57:53.563 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.558+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face019fa50 con 0x7face00ff770 2023-12-12T09:57:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.559+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face0108a90 con 0x7face00ff770 2023-12-12T09:57:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.560+0000 7facd77fe700 1 -- 172.21.15.42:0/2677835182 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7facd0004380 con 0x7face00ff770 2023-12-12T09:57:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.561+0000 7facd77fe700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faccc0a1510 0x7faccc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.561+0000 7face5a0a700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faccc0a1510 0x7faccc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.561+0000 7facd77fe700 1 -- 172.21.15.42:0/2677835182 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7facd00ccd60 con 0x7face00ff770 2023-12-12T09:57:53.564 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.561+0000 7face5a0a700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faccc0a1510 0x7faccc0a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7facdc00b5c0 tx=0x7facdc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:53.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.565+0000 7facd77fe700 1 -- 172.21.15.42:0/2677835182 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7facd0095f50 con 0x7face00ff770 2023-12-12T09:57:53.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:53 smithi042 ceph-mon[98894]: pgmap v301: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:57:53.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:53 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3800035407' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-12T09:57:53.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:53 smithi111 ceph-mon[102565]: pgmap v301: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:57:53.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:53 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3800035407' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-12T09:57:53.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.771+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7face00679a0 con 0x7face00ff770 2023-12-12T09:57:53.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.773+0000 7facd77fe700 1 -- 172.21.15.42:0/2677835182 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24476 (secure 0 0 0) 0x7facd0099570 con 0x7face00ff770 2023-12-12T09:57:53.776 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:53.776 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:57:29.362715+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:57:53.778 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faccc0a1510 msgr2=0x7faccc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.778 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faccc0a1510 0x7faccc0a39d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7facdc00b5c0 tx=0x7facdc005fd0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.778 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face00ff770 msgr2=0x7face0197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.778 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face00ff770 0x7face0197460 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7facd000c970 tx=0x7facd000cc80 comp rx=0 tx=0).stop 2023-12-12T09:57:53.778 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 shutdown_connections 2023-12-12T09:57:53.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faccc0a1510 0x7faccc0a39d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7face0103180 0x7face019e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7face00ff770 0x7face0197460 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 --2- 172.21.15.42:0/2677835182 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7face01027e0 0x7face01979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 >> 172.21.15.42:0/2677835182 conn(0x7face00fb0a0 msgr2=0x7face0105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:53.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 shutdown_connections 2023-12-12T09:57:53.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.776+0000 7face846f700 1 -- 172.21.15.42:0/2677835182 wait complete. 2023-12-12T09:57:53.791 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:56:35.884237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:56:37.283813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:56:40.918865+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:57:53.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:57:53.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:53 smithi173 ceph-mon[103308]: pgmap v301: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:57:53.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:53 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3800035407' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.919+0000 7feb4925e700 1 -- 172.21.15.42:0/2911706779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 msgr2=0x7feb441088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.919+0000 7feb4925e700 1 --2- 172.21.15.42:0/2911706779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 0x7feb441088c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00b0d0 tx=0x7feb3c00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.919+0000 7feb4925e700 1 -- 172.21.15.42:0/2911706779 shutdown_connections 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.919+0000 7feb4925e700 1 --2- 172.21.15.42:0/2911706779 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb44100480 0x7feb441008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.919+0000 7feb4925e700 1 --2- 172.21.15.42:0/2911706779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 0x7feb441088c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.919+0000 7feb4925e700 1 --2- 172.21.15.42:0/2911706779 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb44101670 0x7feb44101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.919+0000 7feb4925e700 1 -- 172.21.15.42:0/2911706779 >> 172.21.15.42:0/2911706779 conn(0x7feb440fb9f0 msgr2=0x7feb440fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.919+0000 7feb4925e700 1 -- 172.21.15.42:0/2911706779 shutdown_connections 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.919+0000 7feb4925e700 1 -- 172.21.15.42:0/2911706779 wait complete. 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4925e700 1 Processor -- start 2023-12-12T09:57:53.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4925e700 1 -- start start 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4925e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb44100480 0x7feb441043c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4925e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb44101670 0x7feb44102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4925e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 0x7feb44102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4925e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb44103730 con 0x7feb44102010 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4925e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb441038a0 con 0x7feb44101670 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4925e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb441a1650 con 0x7feb44100480 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 0x7feb44102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 0x7feb44102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34486/0 (socket says 172.21.15.42:34486) 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4359e700 1 -- 172.21.15.42:0/1180423613 learned_addr learned my addr 172.21.15.42:0/1180423613 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb4259c700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb44101670 0x7feb44102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:53.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.920+0000 7feb42d9d700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb44100480 0x7feb441043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb4359e700 1 -- 172.21.15.42:0/1180423613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb44100480 msgr2=0x7feb441043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb4359e700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb44100480 0x7feb441043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb4359e700 1 -- 172.21.15.42:0/1180423613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb44101670 msgr2=0x7feb44102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb4359e700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb44101670 0x7feb44102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb4359e700 1 -- 172.21.15.42:0/1180423613 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c009d00 con 0x7feb44102010 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb42d9d700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb44100480 0x7feb441043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb4359e700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 0x7feb44102f90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feb3c007b60 tx=0x7feb3c008f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb2bfff700 1 -- 172.21.15.42:0/1180423613 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00e040 con 0x7feb44102010 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb2bfff700 1 -- 172.21.15.42:0/1180423613 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb3c00be30 con 0x7feb44102010 2023-12-12T09:57:53.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb441a1880 con 0x7feb44102010 2023-12-12T09:57:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb2bfff700 1 -- 172.21.15.42:0/1180423613 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c01bae0 con 0x7feb44102010 2023-12-12T09:57:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.921+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb441a1c50 con 0x7feb44102010 2023-12-12T09:57:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.922+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb440682f0 con 0x7feb44102010 2023-12-12T09:57:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.923+0000 7feb2bfff700 1 -- 172.21.15.42:0/1180423613 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7feb3c01d670 con 0x7feb44102010 2023-12-12T09:57:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.924+0000 7feb2bfff700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7feb2c0a1510 0x7feb2c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:53.927 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.924+0000 7feb42d9d700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7feb2c0a1510 0x7feb2c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.924+0000 7feb2bfff700 1 -- 172.21.15.42:0/1180423613 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feb3c0cf190 con 0x7feb44102010 2023-12-12T09:57:53.928 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.924+0000 7feb42d9d700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7feb2c0a1510 0x7feb2c0a39d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb441014d0 tx=0x7feb3400b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:53.929 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:53.927+0000 7feb2bfff700 1 -- 172.21.15.42:0/1180423613 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7feb3c0983b0 con 0x7feb44102010 2023-12-12T09:57:54.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.129+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb4410be30 con 0x7feb44102010 2023-12-12T09:57:54.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.130+0000 7feb2bfff700 1 -- 172.21.15.42:0/1180423613 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb3c09b9d0 con 0x7feb44102010 2023-12-12T09:57:54.132 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T09:57:54.134 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7feb2c0a1510 msgr2=0x7feb2c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7feb2c0a1510 0x7feb2c0a39d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feb441014d0 tx=0x7feb3400b670 comp rx=0 tx=0).stop 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 msgr2=0x7feb44102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 0x7feb44102f90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feb3c007b60 tx=0x7feb3c008f10 comp rx=0 tx=0).stop 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 shutdown_connections 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7feb2c0a1510 0x7feb2c0a39d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7feb44102010 0x7feb44102f90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feb44100480 0x7feb441043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 --2- 172.21.15.42:0/1180423613 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feb44101670 0x7feb44102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 >> 172.21.15.42:0/1180423613 conn(0x7feb440fb9f0 msgr2=0x7feb440fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 shutdown_connections 2023-12-12T09:57:54.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.132+0000 7feb4925e700 1 -- 172.21.15.42:0/1180423613 wait complete. 2023-12-12T09:57:54.147 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:57:54.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 -- 172.21.15.42:0/2528588040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 msgr2=0x7f8d6c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2528588040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 0x7f8d6c101ae0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0099c0 tx=0x7f8d5c009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 -- 172.21.15.42:0/2528588040 shutdown_connections 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2528588040 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c102020 0x7f8d6c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2528588040 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d6c100490 0x7f8d6c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2528588040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 0x7f8d6c101ae0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 -- 172.21.15.42:0/2528588040 >> 172.21.15.42:0/2528588040 conn(0x7f8d6c0fba20 msgr2=0x7f8d6c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 -- 172.21.15.42:0/2528588040 shutdown_connections 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 -- 172.21.15.42:0/2528588040 wait complete. 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.274+0000 7f8d73de8700 1 Processor -- start 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d73de8700 1 -- start start 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d73de8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c100490 0x7f8d6c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d73de8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 0x7f8d6c196970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d73de8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d6c102020 0x7f8d6c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d73de8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c19df60 con 0x7f8d6c101680 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d73de8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c19e0a0 con 0x7f8d6c100490 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d73de8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c19e230 con 0x7f8d6c102020 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71383700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 0x7f8d6c196970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71383700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 0x7f8d6c196970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34498/0 (socket says 172.21.15.42:34498) 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71383700 1 -- 172.21.15.42:0/2853206340 learned_addr learned my addr 172.21.15.42:0/2853206340 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71b84700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c100490 0x7f8d6c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d72385700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d6c102020 0x7f8d6c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71383700 1 -- 172.21.15.42:0/2853206340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d6c102020 msgr2=0x7f8d6c19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71383700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d6c102020 0x7f8d6c19d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71383700 1 -- 172.21.15.42:0/2853206340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c100490 msgr2=0x7f8d6c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71383700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c100490 0x7f8d6c196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71383700 1 -- 172.21.15.42:0/2853206340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c0095f0 con 0x7f8d6c101680 2023-12-12T09:57:54.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.275+0000 7f8d71383700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 0x7f8d6c196970 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c006010 tx=0x7f8d5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:54.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.276+0000 7f8d62ffd700 1 -- 172.21.15.42:0/2853206340 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c01d070 con 0x7f8d6c101680 2023-12-12T09:57:54.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.276+0000 7f8d62ffd700 1 -- 172.21.15.42:0/2853206340 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d5c003f90 con 0x7f8d6c101680 2023-12-12T09:57:54.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.276+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6c0698d0 con 0x7f8d6c101680 2023-12-12T09:57:54.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.276+0000 7f8d62ffd700 1 -- 172.21.15.42:0/2853206340 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c00fd70 con 0x7f8d6c101680 2023-12-12T09:57:54.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.276+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6c069d80 con 0x7f8d6c101680 2023-12-12T09:57:54.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.277+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6c067a80 con 0x7f8d6c101680 2023-12-12T09:57:54.282 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.278+0000 7f8d62ffd700 1 -- 172.21.15.42:0/2853206340 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f8d5c00f460 con 0x7f8d6c101680 2023-12-12T09:57:54.282 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.278+0000 7f8d62ffd700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d580a1510 0x7f8d580a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.282 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.279+0000 7f8d71b84700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d580a1510 0x7f8d580a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.282 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.279+0000 7f8d62ffd700 1 -- 172.21.15.42:0/2853206340 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8d5c0ce440 con 0x7f8d6c101680 2023-12-12T09:57:54.282 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.279+0000 7f8d71b84700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d580a1510 0x7f8d580a39d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c1014e0 tx=0x7f8d6800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:54.284 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.282+0000 7f8d62ffd700 1 -- 172.21.15.42:0/2853206340 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8d5c0976c0 con 0x7f8d6c101680 2023-12-12T09:57:54.489 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.486+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d6c1072f0 con 0x7f8d6c101680 2023-12-12T09:57:54.489 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.487+0000 7f8d62ffd700 1 -- 172.21.15.42:0/2853206340 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d5c09ace0 con 0x7f8d6c101680 2023-12-12T09:57:54.489 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:57:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d580a1510 msgr2=0x7f8d580a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d580a1510 0x7f8d580a39d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c1014e0 tx=0x7f8d6800b410 comp rx=0 tx=0).stop 2023-12-12T09:57:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 msgr2=0x7f8d6c196970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 0x7f8d6c196970 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c006010 tx=0x7f8d5c004730 comp rx=0 tx=0).stop 2023-12-12T09:57:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 shutdown_connections 2023-12-12T09:57:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8d580a1510 0x7f8d580a39d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.491 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8d6c100490 0x7f8d6c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d6c102020 0x7f8d6c19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 --2- 172.21.15.42:0/2853206340 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d6c101680 0x7f8d6c196970 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 >> 172.21.15.42:0/2853206340 conn(0x7f8d6c0fba20 msgr2=0x7f8d6c104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:54.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 shutdown_connections 2023-12-12T09:57:54.492 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.489+0000 7f8d73de8700 1 -- 172.21.15.42:0/2853206340 wait complete. 2023-12-12T09:57:54.503 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:57:54.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 -- 172.21.15.42:0/2943256521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 msgr2=0x7f2d28102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/2943256521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 0x7f2d28102bf0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0099c0 tx=0x7f2d1c009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 -- 172.21.15.42:0/2943256521 shutdown_connections 2023-12-12T09:57:54.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/2943256521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28103130 0x7f2d281099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/2943256521 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d281015a0 0x7f2d281019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/2943256521 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 0x7f2d28102bf0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 -- 172.21.15.42:0/2943256521 >> 172.21.15.42:0/2943256521 conn(0x7f2d280fcb10 msgr2=0x7f2d280fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:54.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 -- 172.21.15.42:0/2943256521 shutdown_connections 2023-12-12T09:57:54.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 -- 172.21.15.42:0/2943256521 wait complete. 2023-12-12T09:57:54.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.632+0000 7f2d2d5af700 1 Processor -- start 2023-12-12T09:57:54.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d2d5af700 1 -- start start 2023-12-12T09:57:54.637 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d2d5af700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d281015a0 0x7f2d281996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.637 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d2d5af700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 0x7f2d28199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.637 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d2d5af700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28103130 0x7f2d281a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.637 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d2d5af700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281a1220 con 0x7f2d28102790 2023-12-12T09:57:54.637 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d2d5af700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281a1360 con 0x7f2d28103130 2023-12-12T09:57:54.637 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d2d5af700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d281a14f0 con 0x7f2d281015a0 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d267fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 0x7f2d28199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d267fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 0x7f2d28199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34512/0 (socket says 172.21.15.42:34512) 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d267fc700 1 -- 172.21.15.42:0/1162173136 learned_addr learned my addr 172.21.15.42:0/1162173136 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d277fe700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28103130 0x7f2d281a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d26ffd700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d281015a0 0x7f2d281996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d267fc700 1 -- 172.21.15.42:0/1162173136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d281015a0 msgr2=0x7f2d281996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d267fc700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d281015a0 0x7f2d281996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d267fc700 1 -- 172.21.15.42:0/1162173136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28103130 msgr2=0x7f2d281a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d267fc700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28103130 0x7f2d281a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d267fc700 1 -- 172.21.15.42:0/1162173136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1c0095f0 con 0x7f2d28102790 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d26ffd700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d281015a0 0x7f2d281996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.633+0000 7f2d267fc700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 0x7f2d28199c30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0036f0 tx=0x7f2d1c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.634+0000 7f2d0ffff700 1 -- 172.21.15.42:0/1162173136 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c01d070 con 0x7f2d28102790 2023-12-12T09:57:54.638 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.634+0000 7f2d0ffff700 1 -- 172.21.15.42:0/1162173136 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d1c004150 con 0x7f2d28102790 2023-12-12T09:57:54.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.634+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d281a1770 con 0x7f2d28102790 2023-12-12T09:57:54.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.634+0000 7f2d0ffff700 1 -- 172.21.15.42:0/1162173136 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c022e60 con 0x7f2d28102790 2023-12-12T09:57:54.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.634+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d281a1b90 con 0x7f2d28102790 2023-12-12T09:57:54.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.635+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d28068310 con 0x7f2d28102790 2023-12-12T09:57:54.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.636+0000 7f2d0ffff700 1 -- 172.21.15.42:0/1162173136 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2d1c022570 con 0x7f2d28102790 2023-12-12T09:57:54.639 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.636+0000 7f2d0ffff700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2d100a1510 0x7f2d100a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.636+0000 7f2d0ffff700 1 -- 172.21.15.42:0/1162173136 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2d1c0ce460 con 0x7f2d28102790 2023-12-12T09:57:54.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.636+0000 7f2d26ffd700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2d100a1510 0x7f2d100a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.637+0000 7f2d26ffd700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2d100a1510 0x7f2d100a39d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d281025f0 tx=0x7f2d1800a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:54.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.641+0000 7f2d0ffff700 1 -- 172.21.15.42:0/1162173136 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2d1c05d2f0 con 0x7f2d28102790 2023-12-12T09:57:54.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:54 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2677835182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:54.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:54 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1180423613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:57:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:54 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2677835182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:54 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1180423613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:57:54.845 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.843+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d281a1e10 con 0x7f2d28102790 2023-12-12T09:57:54.845 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.843+0000 7f2d0ffff700 1 -- 172.21.15.42:0/1162173136 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d1c0976e0 con 0x7f2d28102790 2023-12-12T09:57:54.846 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.845+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2d100a1510 msgr2=0x7f2d100a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.845+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2d100a1510 0x7f2d100a39d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2d281025f0 tx=0x7f2d1800a680 comp rx=0 tx=0).stop 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.845+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 msgr2=0x7f2d28199c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.845+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 0x7f2d28199c30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c0036f0 tx=0x7f2d1c009580 comp rx=0 tx=0).stop 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.846+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 shutdown_connections 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.846+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2d100a1510 0x7f2d100a39d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.846+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2d28103130 0x7f2d281a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.846+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d281015a0 0x7f2d281996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.846+0000 7f2d2d5af700 1 --2- 172.21.15.42:0/1162173136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d28102790 0x7f2d28199c30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.846+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 >> 172.21.15.42:0/1162173136 conn(0x7f2d280fcb10 msgr2=0x7f2d28073800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:54.848 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.846+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 shutdown_connections 2023-12-12T09:57:54.849 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.846+0000 7f2d2d5af700 1 -- 172.21.15.42:0/1162173136 wait complete. 2023-12-12T09:57:54.859 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:54 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2677835182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:54.859 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:54 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1180423613' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:57:54.860 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:57:54.988 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.986+0000 7f77eaafe700 1 -- 172.21.15.42:0/2928255891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41016a0 msgr2=0x7f77e4101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.988 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.986+0000 7f77eaafe700 1 --2- 172.21.15.42:0/2928255891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41016a0 0x7f77e4101b00 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f77d00099c0 tx=0x7f77d0009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.986+0000 7f77eaafe700 1 -- 172.21.15.42:0/2928255891 shutdown_connections 2023-12-12T09:57:54.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.986+0000 7f77eaafe700 1 --2- 172.21.15.42:0/2928255891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e41004b0 0x7f77e41008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.986+0000 7f77eaafe700 1 --2- 172.21.15.42:0/2928255891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77e4102040 0x7f77e41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.986+0000 7f77eaafe700 1 --2- 172.21.15.42:0/2928255891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41016a0 0x7f77e4101b00 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.986+0000 7f77eaafe700 1 -- 172.21.15.42:0/2928255891 >> 172.21.15.42:0/2928255891 conn(0x7f77e40fba20 msgr2=0x7f77e40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:54.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.986+0000 7f77eaafe700 1 -- 172.21.15.42:0/2928255891 shutdown_connections 2023-12-12T09:57:54.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.986+0000 7f77eaafe700 1 -- 172.21.15.42:0/2928255891 wait complete. 2023-12-12T09:57:54.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77eaafe700 1 Processor -- start 2023-12-12T09:57:54.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77eaafe700 1 -- start start 2023-12-12T09:57:54.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77eaafe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41004b0 0x7f77e4196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77eaafe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e41016a0 0x7f77e4196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77eaafe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77e4102040 0x7f77e419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77eaafe700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e419df50 con 0x7f77e41004b0 2023-12-12T09:57:54.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77eaafe700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e419e090 con 0x7f77e41016a0 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77eaafe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e419e220 con 0x7f77e4102040 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77e889a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41004b0 0x7f77e4196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77e889a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41004b0 0x7f77e4196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34524/0 (socket says 172.21.15.42:34524) 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77e889a700 1 -- 172.21.15.42:0/3552967174 learned_addr learned my addr 172.21.15.42:0/3552967174 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.987+0000 7f77e909b700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77e4102040 0x7f77e419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e3fff700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e41016a0 0x7f77e4196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e889a700 1 -- 172.21.15.42:0/3552967174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77e4102040 msgr2=0x7f77e419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e889a700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77e4102040 0x7f77e419d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e889a700 1 -- 172.21.15.42:0/3552967174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e41016a0 msgr2=0x7f77e4196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e889a700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e41016a0 0x7f77e4196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e889a700 1 -- 172.21.15.42:0/3552967174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00095f0 con 0x7f77e41004b0 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e889a700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41004b0 0x7f77e4196420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d980 tx=0x7f77d800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e1ffb700 1 -- 172.21.15.42:0/3552967174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d800de60 con 0x7f77e41004b0 2023-12-12T09:57:54.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e419e500 con 0x7f77e41004b0 2023-12-12T09:57:54.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e1ffb700 1 -- 172.21.15.42:0/3552967174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77d80044e0 con 0x7f77e41004b0 2023-12-12T09:57:54.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77e1ffb700 1 -- 172.21.15.42:0/3552967174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8003be0 con 0x7f77e41004b0 2023-12-12T09:57:54.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.988+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e419ea10 con 0x7f77e41004b0 2023-12-12T09:57:54.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.989+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e41078e0 con 0x7f77e41004b0 2023-12-12T09:57:54.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.990+0000 7f77e1ffb700 1 -- 172.21.15.42:0/3552967174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f77d8009bb0 con 0x7f77e41004b0 2023-12-12T09:57:54.994 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.990+0000 7f77e1ffb700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f77d40a1510 0x7f77d40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:54.994 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.991+0000 7f77e1ffb700 1 -- 172.21.15.42:0/3552967174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f77d80ccf30 con 0x7f77e41004b0 2023-12-12T09:57:54.994 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.991+0000 7f77e3fff700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f77d40a1510 0x7f77d40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:54.994 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.991+0000 7f77e3fff700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f77d40a1510 0x7f77d40a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009fd0 tx=0x7f77d000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:54.997 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:54.995+0000 7f77e1ffb700 1 -- 172.21.15.42:0/3552967174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f77d8096120 con 0x7f77e41004b0 2023-12-12T09:57:55.199 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.197+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f77e410bea0 con 0x7f77e41004b0 2023-12-12T09:57:55.200 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.198+0000 7f77e1ffb700 1 -- 172.21.15.42:0/3552967174 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f77d8009940 con 0x7f77e41004b0 2023-12-12T09:57:55.200 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:57:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f77d40a1510 msgr2=0x7f77d40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.202 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f77d40a1510 0x7f77d40a39d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f77d0009fd0 tx=0x7f77d000b540 comp rx=0 tx=0).stop 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41004b0 msgr2=0x7f77e4196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41004b0 0x7f77e4196420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f77d800d980 tx=0x7f77d800dd40 comp rx=0 tx=0).stop 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 shutdown_connections 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f77d40a1510 0x7f77d40a39d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77e4102040 0x7f77e419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77e41004b0 0x7f77e4196420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 --2- 172.21.15.42:0/3552967174 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77e41016a0 0x7f77e4196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 >> 172.21.15.42:0/3552967174 conn(0x7f77e40fba20 msgr2=0x7f77e4104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 shutdown_connections 2023-12-12T09:57:55.203 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.200+0000 7f77eaafe700 1 -- 172.21.15.42:0/3552967174 wait complete. 2023-12-12T09:57:55.215 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:57:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.338+0000 7f6b7797c700 1 -- 172.21.15.42:0/851567639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101570 msgr2=0x7f6b701019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.338+0000 7f6b7797c700 1 --2- 172.21.15.42:0/851567639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101570 0x7f6b701019d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6b600099c0 tx=0x7f6b60009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.338+0000 7f6b7797c700 1 -- 172.21.15.42:0/851567639 shutdown_connections 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.338+0000 7f6b7797c700 1 --2- 172.21.15.42:0/851567639 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b70100380 0x7f6b701007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.338+0000 7f6b7797c700 1 --2- 172.21.15.42:0/851567639 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b70101f10 0x7f6b701087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.338+0000 7f6b7797c700 1 --2- 172.21.15.42:0/851567639 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101570 0x7f6b701019d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.338+0000 7f6b7797c700 1 -- 172.21.15.42:0/851567639 >> 172.21.15.42:0/851567639 conn(0x7f6b700fb8f0 msgr2=0x7f6b700fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.338+0000 7f6b7797c700 1 -- 172.21.15.42:0/851567639 shutdown_connections 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.338+0000 7f6b7797c700 1 -- 172.21.15.42:0/851567639 wait complete. 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b7797c700 1 Processor -- start 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b7797c700 1 -- start start 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b7797c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b70100380 0x7f6b701042c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b7797c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b70101570 0x7f6b70102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b7797c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101f10 0x7f6b70102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b7797c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b70103630 con 0x7f6b70101f10 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b7797c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b701037a0 con 0x7f6b70101570 2023-12-12T09:57:55.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b7797c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b701a1480 con 0x7f6b70100380 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b75f19700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101f10 0x7f6b70102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b75f19700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101f10 0x7f6b70102e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34542/0 (socket says 172.21.15.42:34542) 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b75f19700 1 -- 172.21.15.42:0/1511456566 learned_addr learned my addr 172.21.15.42:0/1511456566 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b75718700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b70100380 0x7f6b701042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.339+0000 7f6b74f17700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b70101570 0x7f6b70102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b75f19700 1 -- 172.21.15.42:0/1511456566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b70100380 msgr2=0x7f6b701042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b75f19700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b70100380 0x7f6b701042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b75f19700 1 -- 172.21.15.42:0/1511456566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b70101570 msgr2=0x7f6b70102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b75f19700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b70101570 0x7f6b70102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b75f19700 1 -- 172.21.15.42:0/1511456566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600095f0 con 0x7f6b70101f10 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b74f17700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b70101570 0x7f6b70102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b75f19700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101f10 0x7f6b70102e90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800ef30 tx=0x7f6b6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b667fc700 1 -- 172.21.15.42:0/1511456566 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b68018070 con 0x7f6b70101f10 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b667fc700 1 -- 172.21.15.42:0/1511456566 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b6800f040 con 0x7f6b70101f10 2023-12-12T09:57:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b701a16b0 con 0x7f6b70101f10 2023-12-12T09:57:55.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b667fc700 1 -- 172.21.15.42:0/1511456566 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800cd50 con 0x7f6b70101f10 2023-12-12T09:57:55.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.340+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b701a1b70 con 0x7f6b70101f10 2023-12-12T09:57:55.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.341+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b70068230 con 0x7f6b70101f10 2023-12-12T09:57:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.342+0000 7f6b667fc700 1 -- 172.21.15.42:0/1511456566 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f6b68003a00 con 0x7f6b70101f10 2023-12-12T09:57:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.342+0000 7f6b667fc700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6b5c0a1510 0x7f6b5c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.343+0000 7f6b667fc700 1 -- 172.21.15.42:0/1511456566 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6b680ccd80 con 0x7f6b70101f10 2023-12-12T09:57:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.343+0000 7f6b75718700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6b5c0a1510 0x7f6b5c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:55.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.343+0000 7f6b75718700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6b5c0a1510 0x7f6b5c0a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b701013d0 tx=0x7f6b6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:55.349 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.347+0000 7f6b667fc700 1 -- 172.21.15.42:0/1511456566 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6b68096000 con 0x7f6b70101f10 2023-12-12T09:57:55.599 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.597+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b7010bbf0 con 0x7f6b70101f10 2023-12-12T09:57:55.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.598+0000 7f6b667fc700 1 -- 172.21.15.42:0/1511456566 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f6b7010bbf0 con 0x7f6b70101f10 2023-12-12T09:57:55.600 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:55.600 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T09:57:07.293702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24487,"mds_3":24482,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:57:55.602 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6b5c0a1510 msgr2=0x7f6b5c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6b5c0a1510 0x7f6b5c0a39d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6b701013d0 tx=0x7f6b6c00b410 comp rx=0 tx=0).stop 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101f10 msgr2=0x7f6b70102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101f10 0x7f6b70102e90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6b6800ef30 tx=0x7f6b6800c3a0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 shutdown_connections 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6b5c0a1510 0x7f6b5c0a39d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b70101f10 0x7f6b70102e90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b70100380 0x7f6b701042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 --2- 172.21.15.42:0/1511456566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b70101570 0x7f6b70102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 >> 172.21.15.42:0/1511456566 conn(0x7f6b700fb8f0 msgr2=0x7f6b700fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 shutdown_connections 2023-12-12T09:57:55.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.600+0000 7f6b7797c700 1 -- 172.21.15.42:0/1511456566 wait complete. 2023-12-12T09:57:55.604 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2023-12-12T09:57:55.614 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:55 smithi042 ceph-mon[98894]: pgmap v302: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:57:55.614 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:55 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2853206340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:57:55.614 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:55 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1162173136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:57:55.614 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:55 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3552967174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:57:55.615 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:57:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.739+0000 7f71319f0700 1 -- 172.21.15.42:0/2561058226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 msgr2=0x7f712c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.739+0000 7f71319f0700 1 --2- 172.21.15.42:0/2561058226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 0x7f712c1007a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f71140099c0 tx=0x7f7114009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.739+0000 7f71319f0700 1 -- 172.21.15.42:0/2561058226 shutdown_connections 2023-12-12T09:57:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.739+0000 7f71319f0700 1 --2- 172.21.15.42:0/2561058226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c101f10 0x7f712c1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.739+0000 7f71319f0700 1 --2- 172.21.15.42:0/2561058226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 0x7f712c1007a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.739+0000 7f71319f0700 1 --2- 172.21.15.42:0/2561058226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f712c101570 0x7f712c1019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.739+0000 7f71319f0700 1 -- 172.21.15.42:0/2561058226 >> 172.21.15.42:0/2561058226 conn(0x7f712c0fb8f0 msgr2=0x7f712c0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.739+0000 7f71319f0700 1 -- 172.21.15.42:0/2561058226 shutdown_connections 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.739+0000 7f71319f0700 1 -- 172.21.15.42:0/2561058226 wait complete. 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f71319f0700 1 Processor -- start 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f71319f0700 1 -- start start 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f71319f0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 0x7f712c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f71319f0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c101570 0x7f712c199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f71319f0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f712c101f10 0x7f712c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f71319f0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1a1140 con 0x7f712c100380 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f71319f0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1a1280 con 0x7f712c101f10 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f71319f0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1a1410 con 0x7f712c101570 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 0x7f712c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 0x7f712c199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34552/0 (socket says 172.21.15.42:34552) 2023-12-12T09:57:55.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712affd700 1 -- 172.21.15.42:0/3137295376 learned_addr learned my addr 172.21.15.42:0/3137295376 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712a7fc700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c101570 0x7f712c199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712b7fe700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f712c101f10 0x7f712c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712affd700 1 -- 172.21.15.42:0/3137295376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c101570 msgr2=0x7f712c199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712affd700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c101570 0x7f712c199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712affd700 1 -- 172.21.15.42:0/3137295376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f712c101f10 msgr2=0x7f712c1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712affd700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f712c101f10 0x7f712c1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712affd700 1 -- 172.21.15.42:0/3137295376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71140095f0 con 0x7f712c100380 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.740+0000 7f712b7fe700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f712c101f10 0x7f712c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.741+0000 7f712affd700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 0x7f712c199610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7114003820 tx=0x7f7114004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.741+0000 7f71309ee700 1 -- 172.21.15.42:0/3137295376 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711401d070 con 0x7f712c100380 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.741+0000 7f71309ee700 1 -- 172.21.15.42:0/3137295376 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71140054f0 con 0x7f712c100380 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.741+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c1a1690 con 0x7f712c100380 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.741+0000 7f71309ee700 1 -- 172.21.15.42:0/3137295376 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7114004a00 con 0x7f712c100380 2023-12-12T09:57:55.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.741+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c1a1b90 con 0x7f712c100380 2023-12-12T09:57:55.746 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.742+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c068230 con 0x7f712c100380 2023-12-12T09:57:55.746 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.743+0000 7f71309ee700 1 -- 172.21.15.42:0/3137295376 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7114003b50 con 0x7f712c100380 2023-12-12T09:57:55.746 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.743+0000 7f71309ee700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f71180a1590 0x7f71180a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.744+0000 7f712a7fc700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f71180a1590 0x7f71180a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.744+0000 7f71309ee700 1 -- 172.21.15.42:0/3137295376 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f71140cdff0 con 0x7f712c100380 2023-12-12T09:57:55.747 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.744+0000 7f712a7fc700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f71180a1590 0x7f71180a3a50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f711c007c60 tx=0x7f711c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:55.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:55 smithi111 ceph-mon[102565]: pgmap v302: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:57:55.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:55 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2853206340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:57:55.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:55 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1162173136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:57:55.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:55 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3552967174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:57:55.749 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.748+0000 7f71309ee700 1 -- 172.21.15.42:0/3137295376 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7114097210 con 0x7f712c100380 2023-12-12T09:57:55.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:55 smithi173 ceph-mon[103308]: pgmap v302: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:57:55.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:55 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2853206340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:57:55.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:55 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1162173136' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:57:55.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:55 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3552967174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:57:55.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.951+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f712c10bb90 con 0x7f712c100380 2023-12-12T09:57:55.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.953+0000 7f71309ee700 1 -- 172.21.15.42:0/3137295376 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24476 (secure 0 0 0) 0x7f711409a830 con 0x7f712c100380 2023-12-12T09:57:55.955 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:55.955 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:57:29.362715+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:57:55.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f71180a1590 msgr2=0x7f71180a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f71180a1590 0x7f71180a3a50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f711c007c60 tx=0x7f711c0073d0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 msgr2=0x7f712c199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:55.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 0x7f712c199610 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7114003820 tx=0x7f7114004730 comp rx=0 tx=0).stop 2023-12-12T09:57:55.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 shutdown_connections 2023-12-12T09:57:55.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f71180a1590 0x7f71180a3a50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f712c101f10 0x7f712c1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f712c100380 0x7f712c199610 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.958 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 --2- 172.21.15.42:0/3137295376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f712c101570 0x7f712c199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:55.958 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 >> 172.21.15.42:0/3137295376 conn(0x7f712c0fb8f0 msgr2=0x7f712c068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:55.958 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 shutdown_connections 2023-12-12T09:57:55.958 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:55.955+0000 7f71319f0700 1 -- 172.21.15.42:0/3137295376 wait complete. 2023-12-12T09:57:55.970 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:56:35.884237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:56:37.283813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:56:40.918865+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:57:55.970 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:57:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.097+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3312777369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48102010 msgr2=0x7f5c481088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.097+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3312777369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48102010 0x7f5c481088c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000b0d0 tx=0x7f5c4000b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.097+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3312777369 shutdown_connections 2023-12-12T09:57:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.097+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3312777369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48102010 0x7f5c481088c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.097+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3312777369 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c48100480 0x7f5c481008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.097+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3312777369 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c48101670 0x7f5c48101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.097+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3312777369 >> 172.21.15.42:0/3312777369 conn(0x7f5c480fb9f0 msgr2=0x7f5c480fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.097+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3312777369 shutdown_connections 2023-12-12T09:57:56.100 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.097+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3312777369 wait complete. 2023-12-12T09:57:56.102 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.098+0000 7f5c4ce76700 1 Processor -- start 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.098+0000 7f5c4ce76700 1 -- start start 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.098+0000 7f5c4ce76700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c48100480 0x7f5c48197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.098+0000 7f5c4ce76700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48101670 0x7f5c48197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.098+0000 7f5c4ce76700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c48102010 0x7f5c4819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.098+0000 7f5c4ce76700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4819f180 con 0x7f5c48101670 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.098+0000 7f5c4ce76700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4819f2c0 con 0x7f5c48100480 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.098+0000 7f5c4ce76700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4819f450 con 0x7f5c48102010 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.098+0000 7f5c45d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48101670 0x7f5c48197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c45d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48101670 0x7f5c48197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34568/0 (socket says 172.21.15.42:34568) 2023-12-12T09:57:56.106 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c45d9b700 1 -- 172.21.15.42:0/3618907166 learned_addr learned my addr 172.21.15.42:0/3618907166 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c46d9d700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c48102010 0x7f5c4819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c4659c700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c48100480 0x7f5c48197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c45d9b700 1 -- 172.21.15.42:0/3618907166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c48102010 msgr2=0x7f5c4819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c45d9b700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c48102010 0x7f5c4819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c45d9b700 1 -- 172.21.15.42:0/3618907166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c48100480 msgr2=0x7f5c48197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c45d9b700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c48100480 0x7f5c48197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c45d9b700 1 -- 172.21.15.42:0/3618907166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c40009d00 con 0x7f5c48101670 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c4659c700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c48100480 0x7f5c48197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c45d9b700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48101670 0x7f5c48197b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e9a0 tx=0x7f5c3800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c377fe700 1 -- 172.21.15.42:0/3618907166 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38009790 con 0x7f5c48101670 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c377fe700 1 -- 172.21.15.42:0/3618907166 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c3800cbd0 con 0x7f5c48101670 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4819f730 con 0x7f5c48101670 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c377fe700 1 -- 172.21.15.42:0/3618907166 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38018680 con 0x7f5c48101670 2023-12-12T09:57:56.107 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.099+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4819fc40 con 0x7f5c48101670 2023-12-12T09:57:56.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.100+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c48068310 con 0x7f5c48101670 2023-12-12T09:57:56.108 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.102+0000 7f5c377fe700 1 -- 172.21.15.42:0/3618907166 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f5c3800cd40 con 0x7f5c48101670 2023-12-12T09:57:56.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.102+0000 7f5c377fe700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c300a1510 0x7f5c300a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.103+0000 7f5c377fe700 1 -- 172.21.15.42:0/3618907166 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c38014070 con 0x7f5c48101670 2023-12-12T09:57:56.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.103+0000 7f5c4659c700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c300a1510 0x7f5c300a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.103+0000 7f5c4659c700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c300a1510 0x7f5c300a39d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c481014d0 tx=0x7f5c3c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:56.109 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.106+0000 7f5c377fe700 1 -- 172.21.15.42:0/3618907166 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f5c380968e0 con 0x7f5c48101670 2023-12-12T09:57:56.310 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.308+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c4819ff00 con 0x7f5c48101670 2023-12-12T09:57:56.310 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.308+0000 7f5c377fe700 1 -- 172.21.15.42:0/3618907166 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c38099f00 con 0x7f5c48101670 2023-12-12T09:57:56.310 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.310+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c300a1510 msgr2=0x7f5c300a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.310+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c300a1510 0x7f5c300a39d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c481014d0 tx=0x7f5c3c009260 comp rx=0 tx=0).stop 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.310+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48101670 msgr2=0x7f5c48197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.310+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48101670 0x7f5c48197b90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5c3800e9a0 tx=0x7f5c3800ecb0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.310+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 shutdown_connections 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.310+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f5c300a1510 0x7f5c300a39d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.310+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5c48100480 0x7f5c48197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.310+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c48102010 0x7f5c4819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.310+0000 7f5c4ce76700 1 --2- 172.21.15.42:0/3618907166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c48101670 0x7f5c48197b90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.311+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 >> 172.21.15.42:0/3618907166 conn(0x7f5c480fb9f0 msgr2=0x7f5c48068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.311+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 shutdown_connections 2023-12-12T09:57:56.313 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.311+0000 7f5c4ce76700 1 -- 172.21.15.42:0/3618907166 wait complete. 2023-12-12T09:57:56.325 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:57:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.452+0000 7fe24917d700 1 -- 172.21.15.42:0/4055820238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441004b0 msgr2=0x7fe2441008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.452+0000 7fe24917d700 1 --2- 172.21.15.42:0/4055820238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441004b0 0x7fe2441008d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe2380099c0 tx=0x7fe238009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.452+0000 7fe24917d700 1 -- 172.21.15.42:0/4055820238 shutdown_connections 2023-12-12T09:57:56.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.452+0000 7fe24917d700 1 --2- 172.21.15.42:0/4055820238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244102040 0x7fe2441088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.452+0000 7fe24917d700 1 --2- 172.21.15.42:0/4055820238 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441004b0 0x7fe2441008d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.452+0000 7fe24917d700 1 --2- 172.21.15.42:0/4055820238 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2441016a0 0x7fe244101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.452+0000 7fe24917d700 1 -- 172.21.15.42:0/4055820238 >> 172.21.15.42:0/4055820238 conn(0x7fe2440fba20 msgr2=0x7fe2440fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:56.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.452+0000 7fe24917d700 1 -- 172.21.15.42:0/4055820238 shutdown_connections 2023-12-12T09:57:56.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.452+0000 7fe24917d700 1 -- 172.21.15.42:0/4055820238 wait complete. 2023-12-12T09:57:56.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24917d700 1 Processor -- start 2023-12-12T09:57:56.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24917d700 1 -- start start 2023-12-12T09:57:56.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24917d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2441004b0 0x7fe244197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24917d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441016a0 0x7fe244197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24917d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244102040 0x7fe24419ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24917d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24419f050 con 0x7fe2441016a0 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24917d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24419f190 con 0x7fe2441004b0 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24917d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24419f320 con 0x7fe244102040 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441016a0 0x7fe244197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe242d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2441004b0 0x7fe244197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441016a0 0x7fe244197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34582/0 (socket says 172.21.15.42:34582) 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24259c700 1 -- 172.21.15.42:0/2958134620 learned_addr learned my addr 172.21.15.42:0/2958134620 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe242d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2441004b0 0x7fe244197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:37640/0 (socket says 172.21.15.42:37640) 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24359e700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244102040 0x7fe24419ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24259c700 1 -- 172.21.15.42:0/2958134620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244102040 msgr2=0x7fe24419ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24259c700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244102040 0x7fe24419ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24259c700 1 -- 172.21.15.42:0/2958134620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2441004b0 msgr2=0x7fe244197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24259c700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2441004b0 0x7fe244197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.453+0000 7fe24259c700 1 -- 172.21.15.42:0/2958134620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380095f0 con 0x7fe2441016a0 2023-12-12T09:57:56.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.454+0000 7fe24259c700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441016a0 0x7fe244197ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe23400d950 tx=0x7fe23400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:56.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.454+0000 7fe22bfff700 1 -- 172.21.15.42:0/2958134620 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400f890 con 0x7fe2441016a0 2023-12-12T09:57:56.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.454+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24419f600 con 0x7fe2441016a0 2023-12-12T09:57:56.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.454+0000 7fe22bfff700 1 -- 172.21.15.42:0/2958134620 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe234010460 con 0x7fe2441016a0 2023-12-12T09:57:56.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.454+0000 7fe22bfff700 1 -- 172.21.15.42:0/2958134620 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23400e610 con 0x7fe2441016a0 2023-12-12T09:57:56.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.454+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24419fb60 con 0x7fe2441016a0 2023-12-12T09:57:56.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.455+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe244067a80 con 0x7fe2441016a0 2023-12-12T09:57:56.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.456+0000 7fe22bfff700 1 -- 172.21.15.42:0/2958134620 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe2340108d0 con 0x7fe2441016a0 2023-12-12T09:57:56.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.456+0000 7fe22bfff700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe22c0a1510 0x7fe22c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.457+0000 7fe22bfff700 1 -- 172.21.15.42:0/2958134620 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe2340ccf70 con 0x7fe2441016a0 2023-12-12T09:57:56.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.457+0000 7fe242d9d700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe22c0a1510 0x7fe22c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.457+0000 7fe242d9d700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe22c0a1510 0x7fe22c0a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe238003820 tx=0x7fe238005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:56.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.460+0000 7fe22bfff700 1 -- 172.21.15.42:0/2958134620 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe234096160 con 0x7fe2441016a0 2023-12-12T09:57:56.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:56 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1511456566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:56.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:56 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3137295376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:56.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:56 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3618907166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:57:56.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.662+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe24410be50 con 0x7fe2441016a0 2023-12-12T09:57:56.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.662+0000 7fe22bfff700 1 -- 172.21.15.42:0/2958134620 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe234099780 con 0x7fe2441016a0 2023-12-12T09:57:56.664 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.664+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe22c0a1510 msgr2=0x7fe22c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.664+0000 7fe24917d700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe22c0a1510 0x7fe22c0a39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe238003820 tx=0x7fe238005920 comp rx=0 tx=0).stop 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.664+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441016a0 msgr2=0x7fe244197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.664+0000 7fe24917d700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441016a0 0x7fe244197ab0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe23400d950 tx=0x7fe23400dc60 comp rx=0 tx=0).stop 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.665+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 shutdown_connections 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.665+0000 7fe24917d700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe22c0a1510 0x7fe22c0a39d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.665+0000 7fe24917d700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe2441004b0 0x7fe244197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.665+0000 7fe24917d700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244102040 0x7fe24419ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.665+0000 7fe24917d700 1 --2- 172.21.15.42:0/2958134620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe2441016a0 0x7fe244197ab0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.667 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.665+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 >> 172.21.15.42:0/2958134620 conn(0x7fe2440fba20 msgr2=0x7fe2440697b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:56.668 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.665+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 shutdown_connections 2023-12-12T09:57:56.668 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.665+0000 7fe24917d700 1 -- 172.21.15.42:0/2958134620 wait complete. 2023-12-12T09:57:56.679 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:57:56.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:56 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1511456566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:56.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:56 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3137295376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:56.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:56 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3618907166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:57:56.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.807+0000 7f9abab78700 1 -- 172.21.15.42:0/3133328323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab4102040 msgr2=0x7f9ab41088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.807+0000 7f9abab78700 1 --2- 172.21.15.42:0/3133328323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab4102040 0x7f9ab41088f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00b0d0 tx=0x7f9aac00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.807+0000 7f9abab78700 1 -- 172.21.15.42:0/3133328323 shutdown_connections 2023-12-12T09:57:56.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.807+0000 7f9abab78700 1 --2- 172.21.15.42:0/3133328323 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ab41004b0 0x7f9ab41008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.807+0000 7f9abab78700 1 --2- 172.21.15.42:0/3133328323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab4102040 0x7f9ab41088f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.807+0000 7f9abab78700 1 --2- 172.21.15.42:0/3133328323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ab41016a0 0x7f9ab4101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.807+0000 7f9abab78700 1 -- 172.21.15.42:0/3133328323 >> 172.21.15.42:0/3133328323 conn(0x7f9ab40fba20 msgr2=0x7f9ab40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:56.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.807+0000 7f9abab78700 1 -- 172.21.15.42:0/3133328323 shutdown_connections 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.807+0000 7f9abab78700 1 -- 172.21.15.42:0/3133328323 wait complete. 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9abab78700 1 Processor -- start 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9abab78700 1 -- start start 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9abab78700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ab41004b0 0x7f9ab406a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9abab78700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab41016a0 0x7f9ab419c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9abab78700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ab4102040 0x7f9ab419e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9abab78700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab419f140 con 0x7f9ab41016a0 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9abab78700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab419f2b0 con 0x7f9ab4102040 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9abab78700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab419f470 con 0x7f9ab41004b0 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab41016a0 0x7f9ab419c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab41016a0 0x7f9ab419c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34596/0 (socket says 172.21.15.42:34596) 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab3fff700 1 -- 172.21.15.42:0/1571055230 learned_addr learned my addr 172.21.15.42:0/1571055230 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab9115700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ab4102040 0x7f9ab419e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab8914700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ab41004b0 0x7f9ab406a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab3fff700 1 -- 172.21.15.42:0/1571055230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ab41004b0 msgr2=0x7f9ab406a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab3fff700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ab41004b0 0x7f9ab406a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab3fff700 1 -- 172.21.15.42:0/1571055230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ab4102040 msgr2=0x7f9ab419e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab3fff700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ab4102040 0x7f9ab419e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab3fff700 1 -- 172.21.15.42:0/1571055230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac009d00 con 0x7f9ab41016a0 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab3fff700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab41016a0 0x7f9ab419c3b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000e9c0 tx=0x7f9aa000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.808+0000 7f9ab1ffb700 1 -- 172.21.15.42:0/1571055230 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00097b0 con 0x7f9ab41016a0 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.809+0000 7f9ab1ffb700 1 -- 172.21.15.42:0/1571055230 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9aa000ee60 con 0x7f9ab41016a0 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.809+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab419f750 con 0x7f9ab41016a0 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.809+0000 7f9ab1ffb700 1 -- 172.21.15.42:0/1571055230 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00054f0 con 0x7f9ab41016a0 2023-12-12T09:57:56.813 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.809+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab419fc60 con 0x7f9ab41016a0 2023-12-12T09:57:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.810+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab4107860 con 0x7f9ab41016a0 2023-12-12T09:57:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.811+0000 7f9ab1ffb700 1 -- 172.21.15.42:0/1571055230 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9aa0004750 con 0x7f9ab41016a0 2023-12-12T09:57:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.811+0000 7f9ab1ffb700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9aa40a1590 0x7f9aa40a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.812+0000 7f9ab8914700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9aa40a1590 0x7f9aa40a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.812+0000 7f9ab1ffb700 1 -- 172.21.15.42:0/1571055230 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9aa0014070 con 0x7f9ab41016a0 2023-12-12T09:57:56.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.812+0000 7f9ab8914700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9aa40a1590 0x7f9aa40a3a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4101500 tx=0x7f9aa8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:56.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:56.816+0000 7f9ab1ffb700 1 -- 172.21.15.42:0/1571055230 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9aa0096510 con 0x7f9ab41016a0 2023-12-12T09:57:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:56 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1511456566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:56 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3137295376' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:56 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3618907166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.019 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.017+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ab410be40 con 0x7f9ab41016a0 2023-12-12T09:57:57.019 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.017+0000 7f9ab1ffb700 1 -- 172.21.15.42:0/1571055230 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9aa0099b30 con 0x7f9ab41016a0 2023-12-12T09:57:57.019 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9aa40a1590 msgr2=0x7f9aa40a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9aa40a1590 0x7f9aa40a3a50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4101500 tx=0x7f9aa8009260 comp rx=0 tx=0).stop 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab41016a0 msgr2=0x7f9ab419c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab41016a0 0x7f9ab419c3b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000e9c0 tx=0x7f9aa000ecd0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 shutdown_connections 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9aa40a1590 0x7f9aa40a3a50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ab4102040 0x7f9ab419e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ab41004b0 0x7f9ab406a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 --2- 172.21.15.42:0/1571055230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ab41016a0 0x7f9ab419c3b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 >> 172.21.15.42:0/1571055230 conn(0x7f9ab40fba20 msgr2=0x7f9ab41049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:57.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 shutdown_connections 2023-12-12T09:57:57.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.020+0000 7f9abab78700 1 -- 172.21.15.42:0/1571055230 wait complete. 2023-12-12T09:57:57.035 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:57:57.163 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.161+0000 7f99a4b7f700 1 -- 172.21.15.42:0/70173731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0107290 msgr2=0x7f99a0109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.161+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/70173731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0107290 0x7f99a0109700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f999800b0a0 tx=0x7f999800b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.161+0000 7f99a4b7f700 1 -- 172.21.15.42:0/70173731 shutdown_connections 2023-12-12T09:57:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.161+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/70173731 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a0102030 0x7f99a0104420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.161+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/70173731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0107290 0x7f99a0109700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.161+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/70173731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99a0104960 0x7f99a0106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.161+0000 7f99a4b7f700 1 -- 172.21.15.42:0/70173731 >> 172.21.15.42:0/70173731 conn(0x7f99a00fba20 msgr2=0x7f99a00fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.161+0000 7f99a4b7f700 1 -- 172.21.15.42:0/70173731 shutdown_connections 2023-12-12T09:57:57.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.161+0000 7f99a4b7f700 1 -- 172.21.15.42:0/70173731 wait complete. 2023-12-12T09:57:57.166 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f99a4b7f700 1 Processor -- start 2023-12-12T09:57:57.166 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f99a4b7f700 1 -- start start 2023-12-12T09:57:57.166 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f99a4b7f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0102030 0x7f99a0197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f99a4b7f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99a0104960 0x7f99a0197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f99a4b7f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a0107290 0x7f99a019ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f99a4b7f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a019f040 con 0x7f99a0102030 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f99a4b7f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a019f180 con 0x7f99a0107290 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f99a4b7f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a019f310 con 0x7f99a0104960 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f999e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0102030 0x7f99a0197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f999e59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0102030 0x7f99a0197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34608/0 (socket says 172.21.15.42:34608) 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f999e59c700 1 -- 172.21.15.42:0/2888437189 learned_addr learned my addr 172.21.15.42:0/2888437189 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f999dd9b700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99a0104960 0x7f99a0197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.162+0000 7f999ed9d700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a0107290 0x7f99a019ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f999e59c700 1 -- 172.21.15.42:0/2888437189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99a0104960 msgr2=0x7f99a0197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f999e59c700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99a0104960 0x7f99a0197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f999e59c700 1 -- 172.21.15.42:0/2888437189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a0107290 msgr2=0x7f99a019ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f999e59c700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a0107290 0x7f99a019ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f999e59c700 1 -- 172.21.15.42:0/2888437189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9998009d00 con 0x7f99a0102030 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f999dd9b700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99a0104960 0x7f99a0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f999ed9d700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a0107290 0x7f99a019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f999e59c700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0102030 0x7f99a0197510 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f999000c9b0 tx=0x7f999000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f998f7fe700 1 -- 172.21.15.42:0/2888437189 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990007b00 con 0x7f99a0102030 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f998f7fe700 1 -- 172.21.15.42:0/2888437189 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9990007c60 con 0x7f99a0102030 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f998f7fe700 1 -- 172.21.15.42:0/2888437189 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990018660 con 0x7f99a0102030 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a019f5f0 con 0x7f99a0102030 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.163+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a019fb00 con 0x7f99a0102030 2023-12-12T09:57:57.168 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.164+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a00fd5a0 con 0x7f99a0102030 2023-12-12T09:57:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.165+0000 7f998f7fe700 1 -- 172.21.15.42:0/2888437189 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9990018880 con 0x7f99a0102030 2023-12-12T09:57:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.166+0000 7f998f7fe700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99880a1590 0x7f99880a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.166+0000 7f999dd9b700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99880a1590 0x7f99880a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.166+0000 7f998f7fe700 1 -- 172.21.15.42:0/2888437189 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f99900cd1f0 con 0x7f99a0102030 2023-12-12T09:57:57.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.166+0000 7f999dd9b700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99880a1590 0x7f99880a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9994005950 tx=0x7f99940058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:57.172 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.170+0000 7f998f7fe700 1 -- 172.21.15.42:0/2888437189 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9990096380 con 0x7f99a0102030 2023-12-12T09:57:57.374 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.372+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f99a00fcf80 con 0x7f99a0102030 2023-12-12T09:57:57.375 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.373+0000 7f998f7fe700 1 -- 172.21.15.42:0/2888437189 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f99900999a0 con 0x7f99a0102030 2023-12-12T09:57:57.375 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:57:57.377 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99880a1590 msgr2=0x7f99880a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99880a1590 0x7f99880a3a50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9994005950 tx=0x7f99940058e0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0102030 msgr2=0x7f99a0197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0102030 0x7f99a0197510 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f999000c9b0 tx=0x7f999000cd70 comp rx=0 tx=0).stop 2023-12-12T09:57:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 shutdown_connections 2023-12-12T09:57:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99880a1590 0x7f99880a3a50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99a0107290 0x7f99a019ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.378 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99a0102030 0x7f99a0197510 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 --2- 172.21.15.42:0/2888437189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99a0104960 0x7f99a0197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 >> 172.21.15.42:0/2888437189 conn(0x7f99a00fba20 msgr2=0x7f99a0102990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 shutdown_connections 2023-12-12T09:57:57.379 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.375+0000 7f99a4b7f700 1 -- 172.21.15.42:0/2888437189 wait complete. 2023-12-12T09:57:57.390 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:57:57.520 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.515+0000 7f8760682700 1 -- 172.21.15.42:0/275818787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758101570 msgr2=0x7f87581019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.520 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.515+0000 7f8760682700 1 --2- 172.21.15.42:0/275818787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758101570 0x7f87581019d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f87540099c0 tx=0x7f8754009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.520 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.515+0000 7f8760682700 1 -- 172.21.15.42:0/275818787 shutdown_connections 2023-12-12T09:57:57.520 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.515+0000 7f8760682700 1 --2- 172.21.15.42:0/275818787 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8758100380 0x7f87581007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.520 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.515+0000 7f8760682700 1 --2- 172.21.15.42:0/275818787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8758101f10 0x7f87581087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.520 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.515+0000 7f8760682700 1 --2- 172.21.15.42:0/275818787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758101570 0x7f87581019d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.520 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.515+0000 7f8760682700 1 -- 172.21.15.42:0/275818787 >> 172.21.15.42:0/275818787 conn(0x7f87580fb8f0 msgr2=0x7f87580fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:57.520 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.515+0000 7f8760682700 1 -- 172.21.15.42:0/275818787 shutdown_connections 2023-12-12T09:57:57.520 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.515+0000 7f8760682700 1 -- 172.21.15.42:0/275818787 wait complete. 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f8760682700 1 Processor -- start 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f8760682700 1 -- start start 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f8760682700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758100380 0x7f8758069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f8760682700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8758101570 0x7f875819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f8760682700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8758101f10 0x7f875819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f8760682700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875819ef20 con 0x7f8758100380 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f8760682700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875819f090 con 0x7f8758101570 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f8760682700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875819f250 con 0x7f8758101f10 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f875e41e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758100380 0x7f8758069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f875e41e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758100380 0x7f8758069150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34622/0 (socket says 172.21.15.42:34622) 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f875e41e700 1 -- 172.21.15.42:0/1948557084 learned_addr learned my addr 172.21.15.42:0/1948557084 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:57.521 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f875dc1d700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8758101570 0x7f875819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.516+0000 7f875ec1f700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8758101f10 0x7f875819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f875e41e700 1 -- 172.21.15.42:0/1948557084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8758101f10 msgr2=0x7f875819e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f875e41e700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8758101f10 0x7f875819e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f875e41e700 1 -- 172.21.15.42:0/1948557084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8758101570 msgr2=0x7f875819c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f875e41e700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8758101570 0x7f875819c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f875e41e700 1 -- 172.21.15.42:0/1948557084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540095f0 con 0x7f8758100380 2023-12-12T09:57:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f875e41e700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758100380 0x7f8758069150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f874800d980 tx=0x7f874800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f874f7fe700 1 -- 172.21.15.42:0/1948557084 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800f890 con 0x7f8758100380 2023-12-12T09:57:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875819f4d0 con 0x7f8758100380 2023-12-12T09:57:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f874f7fe700 1 -- 172.21.15.42:0/1948557084 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8748010460 con 0x7f8758100380 2023-12-12T09:57:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f874f7fe700 1 -- 172.21.15.42:0/1948557084 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87480195c0 con 0x7f8758100380 2023-12-12T09:57:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.517+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875819fa00 con 0x7f8758100380 2023-12-12T09:57:57.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.518+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8758107800 con 0x7f8758100380 2023-12-12T09:57:57.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.519+0000 7f874f7fe700 1 -- 172.21.15.42:0/1948557084 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f87480105d0 con 0x7f8758100380 2023-12-12T09:57:57.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.520+0000 7f874f7fe700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f87440a1510 0x7f87440a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.520+0000 7f875dc1d700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f87440a1510 0x7f87440a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.520+0000 7f874f7fe700 1 -- 172.21.15.42:0/1948557084 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f87480cd460 con 0x7f8758100380 2023-12-12T09:57:57.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.520+0000 7f875dc1d700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f87440a1510 0x7f87440a39d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8754009990 tx=0x7f8754005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:57.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.523+0000 7f874f7fe700 1 -- 172.21.15.42:0/1948557084 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f87480965f0 con 0x7f8758100380 2023-12-12T09:57:57.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:57 smithi042 ceph-mon[98894]: pgmap v303: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:57:57.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:57 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2958134620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:57 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1571055230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:57 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2888437189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:57 smithi111 ceph-mon[102565]: pgmap v303: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:57:57.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:57 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2958134620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:57 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1571055230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:57 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2888437189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.777 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.775+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8758068230 con 0x7f8758100380 2023-12-12T09:57:57.777 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.775+0000 7f874f7fe700 1 -- 172.21.15.42:0/1948557084 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f8748016090 con 0x7f8758100380 2023-12-12T09:57:57.778 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:57.778 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T09:57:07.293702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24487,"mds_3":24482,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f87440a1510 msgr2=0x7f87440a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f87440a1510 0x7f87440a39d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8754009990 tx=0x7f8754005ad0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758100380 msgr2=0x7f8758069150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758100380 0x7f8758069150 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f874800d980 tx=0x7f874800dc90 comp rx=0 tx=0).stop 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 shutdown_connections 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f87440a1510 0x7f87440a39d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8758101f10 0x7f875819e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8758100380 0x7f8758069150 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 --2- 172.21.15.42:0/1948557084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8758101570 0x7f875819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 >> 172.21.15.42:0/1948557084 conn(0x7f87580fb8f0 msgr2=0x7f8758104940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:57.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.777+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 shutdown_connections 2023-12-12T09:57:57.781 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.778+0000 7f8760682700 1 -- 172.21.15.42:0/1948557084 wait complete. 2023-12-12T09:57:57.781 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2023-12-12T09:57:57.792 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:57:57.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:57 smithi173 ceph-mon[103308]: pgmap v303: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:57:57.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:57 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2958134620' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:57 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1571055230' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:57 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2888437189' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:57:57.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.916+0000 7f2eb894f700 1 -- 172.21.15.42:0/4025661744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4101fc0 msgr2=0x7f2eb41043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.916+0000 7f2eb894f700 1 --2- 172.21.15.42:0/4025661744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4101fc0 0x7f2eb41043b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c009990 tx=0x7f2e9c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.917+0000 7f2eb894f700 1 -- 172.21.15.42:0/4025661744 shutdown_connections 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.917+0000 7f2eb894f700 1 --2- 172.21.15.42:0/4025661744 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2eb4107220 0x7f2eb4109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.917+0000 7f2eb894f700 1 --2- 172.21.15.42:0/4025661744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4101fc0 0x7f2eb41043b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.917+0000 7f2eb894f700 1 --2- 172.21.15.42:0/4025661744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb41048f0 0x7f2eb4106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.917+0000 7f2eb894f700 1 -- 172.21.15.42:0/4025661744 >> 172.21.15.42:0/4025661744 conn(0x7f2eb40fb8f0 msgr2=0x7f2eb40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.917+0000 7f2eb894f700 1 -- 172.21.15.42:0/4025661744 shutdown_connections 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.917+0000 7f2eb894f700 1 -- 172.21.15.42:0/4025661744 wait complete. 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.917+0000 7f2eb894f700 1 Processor -- start 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.917+0000 7f2eb894f700 1 -- start start 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb894f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb4101fc0 0x7f2eb41972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb894f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2eb41048f0 0x7f2eb4197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb894f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4107220 0x7f2eb419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb894f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb419edf0 con 0x7f2eb4107220 2023-12-12T09:57:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb894f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb419ef30 con 0x7f2eb4101fc0 2023-12-12T09:57:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb894f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb419f0c0 con 0x7f2eb41048f0 2023-12-12T09:57:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb2d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4107220 0x7f2eb419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb2d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4107220 0x7f2eb419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34640/0 (socket says 172.21.15.42:34640) 2023-12-12T09:57:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb2d9d700 1 -- 172.21.15.42:0/1694921959 learned_addr learned my addr 172.21.15.42:0/1694921959 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb1d9b700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2eb41048f0 0x7f2eb4197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb259c700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb4101fc0 0x7f2eb41972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb2d9d700 1 -- 172.21.15.42:0/1694921959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2eb41048f0 msgr2=0x7f2eb4197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb2d9d700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2eb41048f0 0x7f2eb4197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb2d9d700 1 -- 172.21.15.42:0/1694921959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb4101fc0 msgr2=0x7f2eb41972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb2d9d700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb4101fc0 0x7f2eb41972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb2d9d700 1 -- 172.21.15.42:0/1694921959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0095f0 con 0x7f2eb4107220 2023-12-12T09:57:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eb2d9d700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4107220 0x7f2eb419e820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2eac007d70 tx=0x7f2eac00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.918+0000 7f2eab7fe700 1 -- 172.21.15.42:0/1694921959 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00ad40 con 0x7f2eb4107220 2023-12-12T09:57:57.924 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.919+0000 7f2eab7fe700 1 -- 172.21.15.42:0/1694921959 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2eac0128e0 con 0x7f2eb4107220 2023-12-12T09:57:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.919+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb419f3a0 con 0x7f2eb4107220 2023-12-12T09:57:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.919+0000 7f2eab7fe700 1 -- 172.21.15.42:0/1694921959 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0116b0 con 0x7f2eb4107220 2023-12-12T09:57:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.919+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb419f8b0 con 0x7f2eb4107220 2023-12-12T09:57:57.925 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.919+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb40fd470 con 0x7f2eb4107220 2023-12-12T09:57:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.921+0000 7f2eab7fe700 1 -- 172.21.15.42:0/1694921959 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2eac012420 con 0x7f2eb4107220 2023-12-12T09:57:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.922+0000 7f2eab7fe700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2ea00a1510 0x7f2ea00a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.922+0000 7f2eb259c700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2ea00a1510 0x7f2ea00a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.922+0000 7f2eab7fe700 1 -- 172.21.15.42:0/1694921959 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2eac0cd6c0 con 0x7f2eb4107220 2023-12-12T09:57:57.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.922+0000 7f2eb259c700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2ea00a1510 0x7f2ea00a39d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00b5c0 tx=0x7f2e9c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:57.927 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:57.925+0000 7f2eab7fe700 1 -- 172.21.15.42:0/1694921959 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2eac0969c0 con 0x7f2eb4107220 2023-12-12T09:57:58.133 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.131+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2eb410cca0 con 0x7f2eb4107220 2023-12-12T09:57:58.135 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.133+0000 7f2eab7fe700 1 -- 172.21.15.42:0/1694921959 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24476 (secure 0 0 0) 0x7f2eac099fe0 con 0x7f2eb4107220 2023-12-12T09:57:58.135 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:58.135 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:57:29.362715+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:57:58.137 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2ea00a1510 msgr2=0x7f2ea00a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2ea00a1510 0x7f2ea00a39d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00b5c0 tx=0x7f2e9c005fb0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4107220 msgr2=0x7f2eb419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4107220 0x7f2eb419e820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2eac007d70 tx=0x7f2eac00a340 comp rx=0 tx=0).stop 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 shutdown_connections 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2ea00a1510 0x7f2ea00a39d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2eb4101fc0 0x7f2eb41972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2eb4107220 0x7f2eb419e820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 --2- 172.21.15.42:0/1694921959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2eb41048f0 0x7f2eb4197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 >> 172.21.15.42:0/1694921959 conn(0x7f2eb40fb8f0 msgr2=0x7f2eb4102900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.138 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 shutdown_connections 2023-12-12T09:57:58.139 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.135+0000 7f2eb894f700 1 -- 172.21.15.42:0/1694921959 wait complete. 2023-12-12T09:57:58.151 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:57:58.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.274+0000 7f6697b62700 1 -- 172.21.15.42:0/1881684186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101570 msgr2=0x7f66901019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.274+0000 7f6697b62700 1 --2- 172.21.15.42:0/1881684186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101570 0x7f66901019d0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f66840099c0 tx=0x7f6684009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.274+0000 7f6697b62700 1 -- 172.21.15.42:0/1881684186 shutdown_connections 2023-12-12T09:57:58.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.274+0000 7f6697b62700 1 --2- 172.21.15.42:0/1881684186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6690101f10 0x7f66901087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.274+0000 7f6697b62700 1 --2- 172.21.15.42:0/1881684186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6690100380 0x7f66901007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.274+0000 7f6697b62700 1 --2- 172.21.15.42:0/1881684186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101570 0x7f66901019d0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.274+0000 7f6697b62700 1 -- 172.21.15.42:0/1881684186 >> 172.21.15.42:0/1881684186 conn(0x7f66900fb8f0 msgr2=0x7f66900fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.274+0000 7f6697b62700 1 -- 172.21.15.42:0/1881684186 shutdown_connections 2023-12-12T09:57:58.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.274+0000 7f6697b62700 1 -- 172.21.15.42:0/1881684186 wait complete. 2023-12-12T09:57:58.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.275+0000 7f6697b62700 1 Processor -- start 2023-12-12T09:57:58.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.275+0000 7f6697b62700 1 -- start start 2023-12-12T09:57:58.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f6697b62700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6690100380 0x7f6690197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f6697b62700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6690101570 0x7f6690197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f6697b62700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101f10 0x7f669019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.279 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f6697b62700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669019ef80 con 0x7f6690101f10 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f6697b62700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669019f0c0 con 0x7f6690100380 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f6697b62700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669019f250 con 0x7f6690101570 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66960ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101f10 0x7f669019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66960ff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101f10 0x7f669019e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34654/0 (socket says 172.21.15.42:34654) 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66950fd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6690101570 0x7f6690197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66960ff700 1 -- 172.21.15.42:0/2931342820 learned_addr learned my addr 172.21.15.42:0/2931342820 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66958fe700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6690100380 0x7f6690197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66960ff700 1 -- 172.21.15.42:0/2931342820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6690101570 msgr2=0x7f6690197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66960ff700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6690101570 0x7f6690197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66960ff700 1 -- 172.21.15.42:0/2931342820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6690100380 msgr2=0x7f6690197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66960ff700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6690100380 0x7f6690197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.276+0000 7f66960ff700 1 -- 172.21.15.42:0/2931342820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66840095f0 con 0x7f6690101f10 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.277+0000 7f66960ff700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101f10 0x7f669019e9b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f668800ada0 tx=0x7f668800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.277+0000 7f6682ffd700 1 -- 172.21.15.42:0/2931342820 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668800f070 con 0x7f6690101f10 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.277+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669019f530 con 0x7f6690101f10 2023-12-12T09:57:58.280 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.277+0000 7f6682ffd700 1 -- 172.21.15.42:0/2931342820 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f668800cd90 con 0x7f6690101f10 2023-12-12T09:57:58.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.277+0000 7f6682ffd700 1 -- 172.21.15.42:0/2931342820 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66880053e0 con 0x7f6690101f10 2023-12-12T09:57:58.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.277+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6690102880 con 0x7f6690101f10 2023-12-12T09:57:58.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.279+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6690068230 con 0x7f6690101f10 2023-12-12T09:57:58.282 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.280+0000 7f6682ffd700 1 -- 172.21.15.42:0/2931342820 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f668800c8d0 con 0x7f6690101f10 2023-12-12T09:57:58.282 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.281+0000 7f6682ffd700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f667c0a5920 0x7f667c0a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.281+0000 7f66958fe700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f667c0a5920 0x7f667c0a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.283 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.282+0000 7f66958fe700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f667c0a5920 0x7f667c0a7de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66901013d0 tx=0x7f668c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.285 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.284+0000 7f6682ffd700 1 -- 172.21.15.42:0/2931342820 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6688004ad0 con 0x7f6690101f10 2023-12-12T09:57:58.288 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.286+0000 7f6682ffd700 1 -- 172.21.15.42:0/2931342820 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f66880952f0 con 0x7f6690101f10 2023-12-12T09:57:58.539 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.537+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f669010bbf0 con 0x7f6690101f10 2023-12-12T09:57:58.540 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.538+0000 7f6682ffd700 1 -- 172.21.15.42:0/2931342820 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f6688098910 con 0x7f6690101f10 2023-12-12T09:57:58.540 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:57:58.540 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T09:57:07.293702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24487,"mds_3":24482,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:57:58.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f667c0a5920 msgr2=0x7f667c0a7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f667c0a5920 0x7f667c0a7de0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66901013d0 tx=0x7f668c00a380 comp rx=0 tx=0).stop 2023-12-12T09:57:58.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101f10 msgr2=0x7f669019e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101f10 0x7f669019e9b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f668800ada0 tx=0x7f668800c330 comp rx=0 tx=0).stop 2023-12-12T09:57:58.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 shutdown_connections 2023-12-12T09:57:58.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f667c0a5920 0x7f667c0a7de0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6690100380 0x7f6690197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.542 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6690101f10 0x7f669019e9b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 --2- 172.21.15.42:0/2931342820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6690101570 0x7f6690197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 >> 172.21.15.42:0/2931342820 conn(0x7f66900fb8f0 msgr2=0x7f6690068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 shutdown_connections 2023-12-12T09:57:58.543 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.540+0000 7f6697b62700 1 -- 172.21.15.42:0/2931342820 wait complete. 2023-12-12T09:57:58.543 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2023-12-12T09:57:58.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-12T09:56:37.902338+0000', 'modified': '2023-12-12T09:57:07.293702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24524, 'mds_1': 24508, 'mds_2': 24487, 'mds_3': 24482, 'mds_4': 24475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.42:6839/229824980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6838', 'nonce': 229824980}, {'type': 'v1', 'addr': '172.21.15.42:6839', 'nonce': 229824980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.42:6841/4221329195', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6840', 'nonce': 4221329195}, {'type': 'v1', 'addr': '172.21.15.42:6841', 'nonce': 4221329195}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6833/1539682179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1539682179}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 1539682179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6833/2859393185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6832', 'nonce': 2859393185}, {'type': 'v1', 'addr': '172.21.15.173:6833', 'nonce': 2859393185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/4127941301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 4127941301}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 4127941301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.173:6835/3087189893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 3087189893}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 3087189893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.111:6837/3946741228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 3946741228}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 3946741228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.173:6837/1667746348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 1667746348}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 1667746348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.173:6839/2977372273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 2977372273}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 2977372273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6839/2430709127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2430709127}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2430709127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-12T09:57:58.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-12-12T09:57:58.555 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.680+0000 7fa23a099700 1 -- 172.21.15.42:0/3592650983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa234102790 msgr2=0x7fa234102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.680+0000 7fa23a099700 1 --2- 172.21.15.42:0/3592650983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa234102790 0x7fa234102bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa2280099c0 tx=0x7fa228009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.680+0000 7fa23a099700 1 -- 172.21.15.42:0/3592650983 shutdown_connections 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.680+0000 7fa23a099700 1 --2- 172.21.15.42:0/3592650983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa234103130 0x7fa2341099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.680+0000 7fa23a099700 1 --2- 172.21.15.42:0/3592650983 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2341015a0 0x7fa2341019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.680+0000 7fa23a099700 1 --2- 172.21.15.42:0/3592650983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa234102790 0x7fa234102bf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.680+0000 7fa23a099700 1 -- 172.21.15.42:0/3592650983 >> 172.21.15.42:0/3592650983 conn(0x7fa2340fcb10 msgr2=0x7fa2340fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.680+0000 7fa23a099700 1 -- 172.21.15.42:0/3592650983 shutdown_connections 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.680+0000 7fa23a099700 1 -- 172.21.15.42:0/3592650983 wait complete. 2023-12-12T09:57:58.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.681+0000 7fa23a099700 1 Processor -- start 2023-12-12T09:57:58.685 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.681+0000 7fa23a099700 1 -- start start 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.681+0000 7fa23a099700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2341015a0 0x7fa234197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.681+0000 7fa23a099700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa234102790 0x7fa234197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.681+0000 7fa23a099700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa234103130 0x7fa23419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.681+0000 7fa23a099700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23419f040 con 0x7fa2341015a0 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.681+0000 7fa23a099700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23419f180 con 0x7fa234103130 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.681+0000 7fa23a099700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23419f310 con 0x7fa234102790 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa2337fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2341015a0 0x7fa234197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa2337fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2341015a0 0x7fa234197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34664/0 (socket says 172.21.15.42:34664) 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa2337fe700 1 -- 172.21.15.42:0/2121637870 learned_addr learned my addr 172.21.15.42:0/2121637870 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa232ffd700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa234102790 0x7fa234197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa233fff700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa234103130 0x7fa23419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa2337fe700 1 -- 172.21.15.42:0/2121637870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa234102790 msgr2=0x7fa234197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa2337fe700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa234102790 0x7fa234197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa2337fe700 1 -- 172.21.15.42:0/2121637870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa234103130 msgr2=0x7fa23419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa2337fe700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa234103130 0x7fa23419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa2337fe700 1 -- 172.21.15.42:0/2121637870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2280095f0 con 0x7fa2341015a0 2023-12-12T09:57:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa233fff700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa234103130 0x7fa23419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa2337fe700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2341015a0 0x7fa234197510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa22400d980 tx=0x7fa22400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa230ff9700 1 -- 172.21.15.42:0/2121637870 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224004020 con 0x7fa2341015a0 2023-12-12T09:57:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa23a099700 1 -- 172.21.15.42:0/2121637870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23419f5f0 con 0x7fa2341015a0 2023-12-12T09:57:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa230ff9700 1 -- 172.21.15.42:0/2121637870 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa224009730 con 0x7fa2341015a0 2023-12-12T09:57:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa230ff9700 1 -- 172.21.15.42:0/2121637870 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224003a30 con 0x7fa2341015a0 2023-12-12T09:57:58.687 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.682+0000 7fa23a099700 1 -- 172.21.15.42:0/2121637870 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23419fb00 con 0x7fa2341015a0 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.684+0000 7fa230ff9700 1 -- 172.21.15.42:0/2121637870 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa2240098a0 con 0x7fa2341015a0 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.685+0000 7fa230ff9700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa21c0a1510 0x7fa21c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.685+0000 7fa230ff9700 1 -- 172.21.15.42:0/2121637870 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa2240cce70 con 0x7fa2341015a0 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.685+0000 7fa232ffd700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa21c0a1510 0x7fa21c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa232ffd700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa21c0a1510 0x7fa21c0a39d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa228006010 tx=0x7fa2280037c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa246292b80 1 Processor -- start 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa246292b80 1 -- start start 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa246292b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6a7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa246292b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.690 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa246292b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6a9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa246292b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f5feb00 con 0x55814f6a80f0 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa246292b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f5fc420 con 0x55814f6a7790 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa246292b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f5f7d10 con 0x55814f6a8fe0 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa232ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa232ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6a8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34674/0 (socket says 172.21.15.42:34674) 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa232ffd700 1 -- 172.21.15.42:0/1235663186 learned_addr learned my addr 172.21.15.42:0/1235663186 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa233fff700 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa2337fe700 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6a7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa232ffd700 1 -- 172.21.15.42:0/1235663186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 msgr2=0x55814f6a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa232ffd700 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa232ffd700 1 -- 172.21.15.42:0/1235663186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 msgr2=0x55814f6a7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa232ffd700 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6a7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa232ffd700 1 -- 172.21.15.42:0/1235663186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55814f4745e0 con 0x55814f6a80f0 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.686+0000 7fa2337fe700 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:58.691 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.687+0000 7fa232ffd700 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6a8a00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa2280099c0 tx=0x7fa228020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6189ce51ba47d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.687+0000 7fa219ffb700 1 -- 172.21.15.42:0/1235663186 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800bdf0 con 0x55814f6a80f0 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.687+0000 7fa219ffb700 1 -- 172.21.15.42:0/1235663186 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22801e040 con 0x55814f6a80f0 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.687+0000 7fa219ffb700 1 -- 172.21.15.42:0/1235663186 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800f880 con 0x55814f6a80f0 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.687+0000 7fa246292b80 1 -- 172.21.15.42:0/1235663186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 msgr2=0x55814f6a8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.687+0000 7fa246292b80 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6a8a00 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa2280099c0 tx=0x7fa228020000 comp rx=0 tx=0).stop 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 -- 172.21.15.42:0/1235663186 shutdown_connections 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6a7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 --2- 172.21.15.42:0/1235663186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6a8a00 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 -- 172.21.15.42:0/1235663186 >> 172.21.15.42:0/1235663186 conn(0x55814f5d2790 msgr2=0x55814f69fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 -- 172.21.15.42:0/1235663186 shutdown_connections 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 -- 172.21.15.42:0/1235663186 wait complete. 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 -- start start 2023-12-12T09:57:58.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 Processor -- start 2023-12-12T09:57:58.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f5b1510 con 0x55814f6a80f0 2023-12-12T09:57:58.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f5b11b0 con 0x55814f6a7790 2023-12-12T09:57:58.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f4768d0 con 0x55814f6a8fe0 2023-12-12T09:57:58.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.688+0000 7fa246292b80 10 client.0 _ll_register_callbacks cb 0x55814f4dca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:58.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa232ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa232ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34680/0 (socket says 172.21.15.42:34680) 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa232ffd700 1 -- 172.21.15.42:0/533519425 learned_addr learned my addr 172.21.15.42:0/533519425 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa233fff700 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa2337fe700 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa232ffd700 1 -- 172.21.15.42:0/533519425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 msgr2=0x55814f6b8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa232ffd700 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6b8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa232ffd700 1 -- 172.21.15.42:0/533519425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 msgr2=0x55814f6b1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa232ffd700 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa232ffd700 1 -- 172.21.15.42:0/533519425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23410a330 con 0x55814f6a80f0 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa2337fe700 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa232ffd700 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6b1a80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa228032040 tx=0x7fa22800f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7174a0997a1b05d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa212ffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa212ffd700 1 -- 172.21.15.42:0/533519425 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228003ac0 con 0x55814f6a80f0 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa212ffd700 1 -- 172.21.15.42:0/533519425 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa228003c20 con 0x55814f6a80f0 2023-12-12T09:57:58.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa246292b80 1 -- 172.21.15.42:0/533519425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55814f696af0 con 0x55814f6a80f0 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa212ffd700 1 -- 172.21.15.42:0/533519425 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa228003de0 con 0x55814f6a80f0 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa246292b80 10 client.15210 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.689+0000 7fa246292b80 1 -- 172.21.15.42:0/533519425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55814f420d90 con 0x55814f6a80f0 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.690+0000 7fa212ffd700 1 -- 172.21.15.42:0/533519425 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55814f696af0 con 0x55814f6a80f0 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.690+0000 7fa212ffd700 1 -- 172.21.15.42:0/533519425 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa228018400 con 0x55814f6a80f0 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.690+0000 7fa246292b80 10 client.15210 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.690+0000 7fa246292b80 10 client.15210 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.690+0000 7fa246292b80 20 client.15210 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.690+0000 7fa246292b80 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x55814f6bcbf0 0x55814f6bba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.690+0000 7fa246292b80 4 client.15210 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.690+0000 7fa246292b80 1 -- 172.21.15.42:0/533519425 --> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55814f672ef0 con 0x55814f6bcbf0 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.690+0000 7fa2337fe700 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x55814f6bcbf0 0x55814f6bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa212ffd700 10 client.15210 ms_handle_connect on v2:172.21.15.173:6834/3087189893 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa2337fe700 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x55814f6bcbf0 0x55814f6bba00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c0005dabb00d2b82 server_cookie=2b6060695b328aca in_seq=0 out_seq=0 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa212ffd700 1 -- 172.21.15.42:0/533519425 <== mds.4 v2:172.21.15.173:6834/3087189893 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55814f672ef0 con 0x55814f6bcbf0 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa212ffd700 10 client.15210 handle_command_reply: tid=0 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa246292b80 1 client.15210 shutdown 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa246292b80 1 -- 172.21.15.42:0/533519425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 msgr2=0x55814f6b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa246292b80 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6b1a80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa228032040 tx=0x7fa22800f8b0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa246292b80 1 -- 172.21.15.42:0/533519425 shutdown_connections 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa246292b80 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.691+0000 7fa246292b80 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x55814f6bcbf0 0x55814f6bba00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.692+0000 7fa246292b80 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6b8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.692+0000 7fa246292b80 1 --2- 172.21.15.42:0/533519425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6b1a80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.692+0000 7fa212ffd700 0 client.15210 ms_handle_reset on v2:172.21.15.173:6834/3087189893 2023-12-12T09:57:58.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.692+0000 7fa246292b80 1 -- 172.21.15.42:0/533519425 >> 172.21.15.42:0/533519425 conn(0x55814f5d2790 msgr2=0x55814f69e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.692+0000 7fa246292b80 1 -- 172.21.15.42:0/533519425 shutdown_connections 2023-12-12T09:57:58.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.692+0000 7fa246292b80 1 -- 172.21.15.42:0/533519425 wait complete. 2023-12-12T09:57:58.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.692+0000 7fa246292b80 20 client.15210 trim_cache size 0 max 16384 2023-12-12T09:57:58.716 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa246292b80 1 Processor -- start 2023-12-12T09:57:58.716 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa246292b80 1 -- start start 2023-12-12T09:57:58.716 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa246292b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.716 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa246292b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.716 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa246292b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa246292b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f5f3e00 con 0x55814f6a80f0 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa246292b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f59cb50 con 0x55814f6a7790 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa246292b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f593360 con 0x55814f6a8fe0 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa232ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa232ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6ac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34688/0 (socket says 172.21.15.42:34688) 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa232ffd700 1 -- 172.21.15.42:0/1793807357 learned_addr learned my addr 172.21.15.42:0/1793807357 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa232ffd700 1 -- 172.21.15.42:0/1793807357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 msgr2=0x55814f6b8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa232ffd700 1 --2- 172.21.15.42:0/1793807357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa232ffd700 1 -- 172.21.15.42:0/1793807357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 msgr2=0x55814f6abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa232ffd700 1 --2- 172.21.15.42:0/1793807357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.710+0000 7fa232ffd700 1 -- 172.21.15.42:0/1793807357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55814f409c10 con 0x55814f6a80f0 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa232ffd700 1 --2- 172.21.15.42:0/1793807357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6ac4c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa2280036f0 tx=0x7fa228009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b1d13ff17ba648 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa21a7fc700 1 -- 172.21.15.42:0/1793807357 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22800ba70 con 0x55814f6a80f0 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa21a7fc700 1 -- 172.21.15.42:0/1793807357 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22800f460 con 0x55814f6a80f0 2023-12-12T09:57:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa21a7fc700 1 -- 172.21.15.42:0/1793807357 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22802f440 con 0x55814f6a80f0 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa246292b80 1 -- 172.21.15.42:0/1793807357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 msgr2=0x55814f6ac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa246292b80 1 --2- 172.21.15.42:0/1793807357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6ac4c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa2280036f0 tx=0x7fa228009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa246292b80 1 -- 172.21.15.42:0/1793807357 shutdown_connections 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa246292b80 1 --2- 172.21.15.42:0/1793807357 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa246292b80 1 --2- 172.21.15.42:0/1793807357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a8fe0 0x55814f6b8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa246292b80 1 --2- 172.21.15.42:0/1793807357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a80f0 0x55814f6ac4c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa246292b80 1 -- 172.21.15.42:0/1793807357 >> 172.21.15.42:0/1793807357 conn(0x55814f5d2790 msgr2=0x55814f6c5550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa246292b80 1 -- 172.21.15.42:0/1793807357 shutdown_connections 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.711+0000 7fa246292b80 1 -- 172.21.15.42:0/1793807357 wait complete. 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa246292b80 1 -- start start 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa246292b80 1 Processor -- start 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa246292b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa246292b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a80f0 0x55814f6b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa246292b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a8fe0 0x55814f6b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.718 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa246292b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f6b60c0 con 0x55814f6a8fe0 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa246292b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f712cb0 con 0x55814f6a7790 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa246292b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814f712df0 con 0x55814f6a80f0 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa246292b80 10 client.0 _ll_register_callbacks cb 0x55814f4dca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa233fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a8fe0 0x55814f6b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa233fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a8fe0 0x55814f6b5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34696/0 (socket says 172.21.15.42:34696) 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa233fff700 1 -- 172.21.15.42:0/3452565643 learned_addr learned my addr 172.21.15.42:0/3452565643 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.712+0000 7fa2337fe700 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa232ffd700 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a80f0 0x55814f6b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa233fff700 1 -- 172.21.15.42:0/3452565643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a80f0 msgr2=0x55814f6b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa233fff700 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a80f0 0x55814f6b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa233fff700 1 -- 172.21.15.42:0/3452565643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 msgr2=0x55814f6b8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa233fff700 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa233fff700 1 -- 172.21.15.42:0/3452565643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55814f420d90 con 0x55814f6a8fe0 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa2337fe700 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:58.719 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa233fff700 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a8fe0 0x55814f6b5ae0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0079f0 tx=0x7fa22c007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2595aa86f38709e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa212ffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa212ffd700 1 -- 172.21.15.42:0/3452565643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c0138d0 con 0x55814f6a8fe0 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa212ffd700 1 -- 172.21.15.42:0/3452565643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa22c013a30 con 0x55814f6a8fe0 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa246292b80 1 -- 172.21.15.42:0/3452565643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa22801e040 con 0x55814f6a8fe0 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa212ffd700 1 -- 172.21.15.42:0/3452565643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa22c013bf0 con 0x55814f6a8fe0 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa246292b80 10 client.15216 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa246292b80 1 -- 172.21.15.42:0/3452565643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55814f447200 con 0x55814f6a8fe0 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.713+0000 7fa212ffd700 1 -- 172.21.15.42:0/3452565643 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa22801e040 con 0x55814f6a8fe0 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa212ffd700 1 -- 172.21.15.42:0/3452565643 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fa22c017430 con 0x55814f6a8fe0 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa246292b80 10 client.15216 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa246292b80 10 client.15216 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa246292b80 20 client.15216 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa246292b80 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x55814f7181b0 0x55814f7145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa246292b80 4 client.15216 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:57:58.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa246292b80 1 -- 172.21.15.42:0/3452565643 --> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55814f5bdf80 con 0x55814f7181b0 2023-12-12T09:57:58.721 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa2337fe700 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x55814f7181b0 0x55814f7145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.721 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa212ffd700 10 client.15216 ms_handle_connect on v2:172.21.15.173:6834/3087189893 2023-12-12T09:57:58.721 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.714+0000 7fa2337fe700 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x55814f7181b0 0x55814f7145f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=39814aac489699a6 server_cookie=c2614eecfb876f62 in_seq=0 out_seq=0 2023-12-12T09:57:58.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa212ffd700 1 -- 172.21.15.42:0/3452565643 <== mds.4 v2:172.21.15.173:6834/3087189893 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55814f5bdf80 con 0x55814f7181b0 2023-12-12T09:57:58.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa212ffd700 10 client.15216 handle_command_reply: tid=0 2023-12-12T09:57:58.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 client.15216 shutdown 2023-12-12T09:57:58.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 -- 172.21.15.42:0/3452565643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a8fe0 msgr2=0x55814f6b5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a8fe0 0x55814f6b5ae0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa22c0079f0 tx=0x7fa22c007db0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 -- 172.21.15.42:0/3452565643 shutdown_connections 2023-12-12T09:57:58.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55814f6a7790 0x55814f6b8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x55814f7181b0 0x55814f7145f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55814f6a8fe0 0x55814f6b5ae0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa212ffd700 0 client.15216 ms_handle_reset on v2:172.21.15.173:6834/3087189893 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 --2- 172.21.15.42:0/3452565643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55814f6a80f0 0x55814f6b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 -- 172.21.15.42:0/3452565643 >> 172.21.15.42:0/3452565643 conn(0x55814f5d2790 msgr2=0x55814f6abf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 -- 172.21.15.42:0/3452565643 shutdown_connections 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 1 -- 172.21.15.42:0/3452565643 wait complete. 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa246292b80 20 client.15216 trim_cache size 0 max 16384 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:57:58.723 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 4, 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24475, 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 52.424926476000003, 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 100.178290308 2023-12-12T09:57:58.724 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:57:58.727 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa23a099700 1 -- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa21c0a1510 msgr2=0x7fa21c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.727 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa23a099700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa21c0a1510 0x7fa21c0a39d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa228006010 tx=0x7fa2280037c0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.727 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa23a099700 1 -- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2341015a0 msgr2=0x7fa234197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.727 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa23a099700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2341015a0 0x7fa234197510 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa22400d980 tx=0x7fa22400dc90 comp rx=0 tx=0).stop 2023-12-12T09:57:58.727 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa23a099700 1 -- 172.21.15.42:0/2121637870 shutdown_connections 2023-12-12T09:57:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa23a099700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa21c0a1510 0x7fa21c0a39d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa23a099700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa234103130 0x7fa23419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa23a099700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2341015a0 0x7fa234197510 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.715+0000 7fa23a099700 1 --2- 172.21.15.42:0/2121637870 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa234102790 0x7fa234197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.716+0000 7fa23a099700 1 -- 172.21.15.42:0/2121637870 >> 172.21.15.42:0/2121637870 conn(0x7fa2340fcb10 msgr2=0x7fa234105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.716+0000 7fa23a099700 1 -- 172.21.15.42:0/2121637870 shutdown_connections 2023-12-12T09:57:58.728 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.716+0000 7fa23a099700 1 -- 172.21.15.42:0/2121637870 wait complete. 2023-12-12T09:57:58.729 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:58 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1948557084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:58.729 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:58 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1694921959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:58.730 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-12-12T09:57:58.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:58 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1948557084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:58.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:58 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1694921959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:58 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1948557084' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:58 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1694921959' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:57:58.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.854+0000 7fb73bcb6700 1 -- 172.21.15.42:0/3370743245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb734102040 msgr2=0x7fb7341088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.854+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/3370743245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb734102040 0x7fb7341088f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00b0d0 tx=0x7fb72c00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.854+0000 7fb73bcb6700 1 -- 172.21.15.42:0/3370743245 shutdown_connections 2023-12-12T09:57:58.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.854+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/3370743245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb7341004b0 0x7fb7341008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.854+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/3370743245 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb734102040 0x7fb7341088f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.854+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/3370743245 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7341016a0 0x7fb734101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.854+0000 7fb73bcb6700 1 -- 172.21.15.42:0/3370743245 >> 172.21.15.42:0/3370743245 conn(0x7fb7340fba20 msgr2=0x7fb7340fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.854+0000 7fb73bcb6700 1 -- 172.21.15.42:0/3370743245 shutdown_connections 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.854+0000 7fb73bcb6700 1 -- 172.21.15.42:0/3370743245 wait complete. 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb73bcb6700 1 Processor -- start 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb73bcb6700 1 -- start start 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb73bcb6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7341004b0 0x7fb734196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb73bcb6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7341016a0 0x7fb7341969d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb73bcb6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb734102040 0x7fb73419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb73bcb6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73419dfc0 con 0x7fb7341004b0 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb73bcb6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73419e100 con 0x7fb734102040 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb73bcb6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73419e290 con 0x7fb7341016a0 2023-12-12T09:57:58.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739a52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7341004b0 0x7fb734196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739a52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7341004b0 0x7fb734196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34706/0 (socket says 172.21.15.42:34706) 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739a52700 1 -- 172.21.15.42:0/255343541 learned_addr learned my addr 172.21.15.42:0/255343541 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb73a253700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb734102040 0x7fb73419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739251700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7341016a0 0x7fb7341969d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739a52700 1 -- 172.21.15.42:0/255343541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7341016a0 msgr2=0x7fb7341969d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739a52700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7341016a0 0x7fb7341969d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739a52700 1 -- 172.21.15.42:0/255343541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb734102040 msgr2=0x7fb73419d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739a52700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb734102040 0x7fb73419d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739a52700 1 -- 172.21.15.42:0/255343541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c009d00 con 0x7fb7341004b0 2023-12-12T09:57:58.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.855+0000 7fb739a52700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7341004b0 0x7fb734196490 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb72400e970 tx=0x7fb72400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.856+0000 7fb72affd700 1 -- 172.21.15.42:0/255343541 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724009790 con 0x7fb7341004b0 2023-12-12T09:57:58.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.856+0000 7fb72affd700 1 -- 172.21.15.42:0/255343541 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb72400eec0 con 0x7fb7341004b0 2023-12-12T09:57:58.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.856+0000 7fb73bcb6700 1 -- 172.21.15.42:0/255343541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb734069910 con 0x7fb7341004b0 2023-12-12T09:57:58.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.856+0000 7fb72affd700 1 -- 172.21.15.42:0/255343541 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724003cf0 con 0x7fb7341004b0 2023-12-12T09:57:58.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.856+0000 7fb73bcb6700 1 -- 172.21.15.42:0/255343541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb734069e70 con 0x7fb7341004b0 2023-12-12T09:57:58.870 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.858+0000 7fb72affd700 1 -- 172.21.15.42:0/255343541 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fb72400ccd0 con 0x7fb7341004b0 2023-12-12T09:57:58.870 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.858+0000 7fb72affd700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb7200a1590 0x7fb7200a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.870 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.859+0000 7fb72affd700 1 -- 172.21.15.42:0/255343541 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb724014070 con 0x7fb7341004b0 2023-12-12T09:57:58.870 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.859+0000 7fb739251700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb7200a1590 0x7fb7200a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.870 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.859+0000 7fb739251700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb7200a1590 0x7fb7200a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb730009bf0 tx=0x7fb730009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.870 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.859+0000 7fb747eafb80 1 Processor -- start 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb747eafb80 1 -- start start 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f3abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f3ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f3c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb747eafb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0e91b00 con 0x55ebe0f3a790 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb747eafb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0e8f420 con 0x55ebe0f3b0f0 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb747eafb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0e8ad10 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739a52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f3abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739a52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f3abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34714/0 (socket says 172.21.15.42:34714) 2023-12-12T09:57:58.871 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739a52700 1 -- 172.21.15.42:0/3038118933 learned_addr learned my addr 172.21.15.42:0/3038118933 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739251700 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f3ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb73a253700 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f3c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739a52700 1 -- 172.21.15.42:0/3038118933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3bfe0 msgr2=0x55ebe0f3c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739a52700 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f3c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739a52700 1 -- 172.21.15.42:0/3038118933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 msgr2=0x55ebe0f3ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739a52700 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f3ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739a52700 1 -- 172.21.15.42:0/3038118933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebe0d075e0 con 0x55ebe0f3a790 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb73a253700 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f3c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb739a52700 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f3abb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7240a20a0 tx=0x7fb724099c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d44a1391485d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.860+0000 7fb71ffff700 1 -- 172.21.15.42:0/3038118933 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72409e170 con 0x55ebe0f3a790 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb71ffff700 1 -- 172.21.15.42:0/3038118933 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb72409a240 con 0x55ebe0f3a790 2023-12-12T09:57:58.872 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb71ffff700 1 -- 172.21.15.42:0/3038118933 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72409a4a0 con 0x55ebe0f3a790 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb747eafb80 1 -- 172.21.15.42:0/3038118933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 msgr2=0x55ebe0f3abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb747eafb80 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f3abb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7240a20a0 tx=0x7fb724099c20 comp rx=0 tx=0).stop 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb747eafb80 1 -- 172.21.15.42:0/3038118933 shutdown_connections 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb747eafb80 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f3c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb747eafb80 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f3abb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb747eafb80 1 --2- 172.21.15.42:0/3038118933 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f3ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb747eafb80 1 -- 172.21.15.42:0/3038118933 >> 172.21.15.42:0/3038118933 conn(0x55ebe0e65790 msgr2=0x55ebe0f32d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb747eafb80 1 -- 172.21.15.42:0/3038118933 shutdown_connections 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.861+0000 7fb747eafb80 1 -- 172.21.15.42:0/3038118933 wait complete. 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb747eafb80 1 -- start start 2023-12-12T09:57:58.873 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb747eafb80 1 Processor -- start 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3a790 0x55ebe0f44590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f44ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb747eafb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0e44510 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb747eafb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0e441b0 con 0x55ebe0f3a790 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb747eafb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0d098d0 con 0x55ebe0f3b0f0 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb747eafb80 10 client.0 _ll_register_callbacks cb 0x55ebe0d6fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb73a253700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb73a253700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34724/0 (socket says 172.21.15.42:34724) 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb73a253700 1 -- 172.21.15.42:0/1985642898 learned_addr learned my addr 172.21.15.42:0/1985642898 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.874 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.862+0000 7fb739a52700 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3a790 0x55ebe0f44590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb739251700 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f44ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb73a253700 1 -- 172.21.15.42:0/1985642898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 msgr2=0x55ebe0f44ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb73a253700 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f44ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb73a253700 1 -- 172.21.15.42:0/1985642898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3a790 msgr2=0x55ebe0f44590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb73a253700 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3a790 0x55ebe0f44590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb73a253700 1 -- 172.21.15.42:0/1985642898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb734069e70 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb73a253700 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4bb10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00b0d0 tx=0x7fb72c009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97c37d64e7e0505 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb71d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb71d7fa700 1 -- 172.21.15.42:0/1985642898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00e040 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb71d7fa700 1 -- 172.21.15.42:0/1985642898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb72c00bca0 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.875 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb747eafb80 1 -- 172.21.15.42:0/1985642898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebe0f29af0 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb71d7fa700 1 -- 172.21.15.42:0/1985642898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb72c00be60 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb747eafb80 10 client.15228 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb747eafb80 1 -- 172.21.15.42:0/1985642898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ebe0cb3d90 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.863+0000 7fb71d7fa700 1 -- 172.21.15.42:0/1985642898 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ebe0f29af0 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb71d7fa700 1 -- 172.21.15.42:0/1985642898 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb72c00b100 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb747eafb80 10 client.15228 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb747eafb80 10 client.15228 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb747eafb80 20 client.15228 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55ebe0f501e0 0x55ebe0f4ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb747eafb80 4 client.15228 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:57:58.876 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb747eafb80 1 -- 172.21.15.42:0/1985642898 --> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebe0f05ef0 con 0x55ebe0f501e0 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb739a52700 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55ebe0f501e0 0x55ebe0f4ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb71d7fa700 10 client.15228 ms_handle_connect on v2:172.21.15.111:6836/3946741228 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.864+0000 7fb739a52700 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55ebe0f501e0 0x55ebe0f4ecb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc747b7f94e10180 server_cookie=a3bdbec66d7b2c61 in_seq=0 out_seq=0 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb71d7fa700 1 -- 172.21.15.42:0/1985642898 <== mds.3 v2:172.21.15.111:6836/3946741228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ebe0f05ef0 con 0x55ebe0f501e0 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb71d7fa700 10 client.15228 handle_command_reply: tid=0 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 client.15228 shutdown 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 -- 172.21.15.42:0/1985642898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 msgr2=0x55ebe0f4bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4bb10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00b0d0 tx=0x7fb72c009000 comp rx=0 tx=0).stop 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 -- 172.21.15.42:0/1985642898 shutdown_connections 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55ebe0f501e0 0x55ebe0f4ecb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3a790 0x55ebe0f44590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.877 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb71d7fa700 0 client.15228 ms_handle_reset on v2:172.21.15.111:6836/3946741228 2023-12-12T09:57:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4bb10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1985642898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f44ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 -- 172.21.15.42:0/1985642898 >> 172.21.15.42:0/1985642898 conn(0x55ebe0e65790 msgr2=0x55ebe0f319f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 -- 172.21.15.42:0/1985642898 shutdown_connections 2023-12-12T09:57:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.865+0000 7fb747eafb80 1 -- 172.21.15.42:0/1985642898 wait complete. 2023-12-12T09:57:58.878 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.866+0000 7fb747eafb80 20 client.15228 trim_cache size 0 max 16384 2023-12-12T09:57:58.891 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.883+0000 7fb747eafb80 1 Processor -- start 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.883+0000 7fb747eafb80 1 -- start start 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f43240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f4ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f3ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb747eafb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0e26360 con 0x55ebe0f3a790 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb747eafb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0e2d830 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb747eafb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0f29ad0 con 0x55ebe0f3b0f0 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739a52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f43240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739a52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f43240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34728/0 (socket says 172.21.15.42:34728) 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739a52700 1 -- 172.21.15.42:0/1708994103 learned_addr learned my addr 172.21.15.42:0/1708994103 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.892 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739251700 1 --2- 172.21.15.42:0/1708994103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f4ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739a52700 1 -- 172.21.15.42:0/1708994103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 msgr2=0x55ebe0f4ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739a52700 1 --2- 172.21.15.42:0/1708994103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f4ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739a52700 1 -- 172.21.15.42:0/1708994103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3bfe0 msgr2=0x55ebe0f3ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739a52700 1 --2- 172.21.15.42:0/1708994103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f3ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739a52700 1 -- 172.21.15.42:0/1708994103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebe0c9cc10 con 0x55ebe0f3a790 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb739a52700 1 --2- 172.21.15.42:0/1708994103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f43240 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb72409a1d0 tx=0x7fb724003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8a79b4a8504134 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb728d82700 1 -- 172.21.15.42:0/1708994103 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240a25d0 con 0x55ebe0f3a790 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb728d82700 1 -- 172.21.15.42:0/1708994103 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb72409e390 con 0x55ebe0f3a790 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.884+0000 7fb728d82700 1 -- 172.21.15.42:0/1708994103 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7240cd230 con 0x55ebe0f3a790 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 -- 172.21.15.42:0/1708994103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 msgr2=0x55ebe0f43240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.893 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1708994103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f43240 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb72409a1d0 tx=0x7fb724003780 comp rx=0 tx=0).stop 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 -- 172.21.15.42:0/1708994103 shutdown_connections 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1708994103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f3ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1708994103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3a790 0x55ebe0f43240 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 --2- 172.21.15.42:0/1708994103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f4ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 -- 172.21.15.42:0/1708994103 >> 172.21.15.42:0/1708994103 conn(0x55ebe0e65790 msgr2=0x55ebe0f42500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 -- 172.21.15.42:0/1708994103 shutdown_connections 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 -- 172.21.15.42:0/1708994103 wait complete. 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 -- start start 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.885+0000 7fb747eafb80 1 Processor -- start 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3a790 0x55ebe0f47ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f48220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.894 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb747eafb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb747eafb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0f4b6d0 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb747eafb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0f4b840 con 0x55ebe0f3b0f0 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb747eafb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebe0f4ba00 con 0x55ebe0f3a790 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb747eafb80 10 client.0 _ll_register_callbacks cb 0x55ebe0d6fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb73a253700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb73a253700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34738/0 (socket says 172.21.15.42:34738) 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb73a253700 1 -- 172.21.15.42:0/533117741 learned_addr learned my addr 172.21.15.42:0/533117741 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb739a52700 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3a790 0x55ebe0f47ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb739251700 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f48220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb73a253700 1 -- 172.21.15.42:0/533117741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3a790 msgr2=0x55ebe0f47ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb73a253700 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3a790 0x55ebe0f47ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb73a253700 1 -- 172.21.15.42:0/533117741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 msgr2=0x55ebe0f48220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb73a253700 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f48220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb73a253700 1 -- 172.21.15.42:0/533117741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7240b6040 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb739251700 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f48220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb73a253700 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4b190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb72c005900 tx=0x7fb72c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdc178055a7fc2a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb71d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb71d7fa700 1 -- 172.21.15.42:0/533117741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00e030 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb71d7fa700 1 -- 172.21.15.42:0/533117741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb72c00f070 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.886+0000 7fb747eafb80 1 -- 172.21.15.42:0/533117741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb72409a240 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb71d7fa700 1 -- 172.21.15.42:0/533117741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb72c01d420 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb747eafb80 10 client.15234 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:58.896 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb747eafb80 1 -- 172.21.15.42:0/533117741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ebe0cda200 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb71d7fa700 1 -- 172.21.15.42:0/533117741 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb72409a240 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb71d7fa700 1 -- 172.21.15.42:0/533117741 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb72c0051a0 con 0x55ebe0f3bfe0 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb747eafb80 10 client.15234 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb747eafb80 10 client.15234 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb747eafb80 20 client.15234 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb747eafb80 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55ebe0faabf0 0x55ebe0fa70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb747eafb80 4 client.15234 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb747eafb80 1 -- 172.21.15.42:0/533117741 --> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ebe0e50f80 con 0x55ebe0faabf0 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.887+0000 7fb739a52700 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55ebe0faabf0 0x55ebe0fa70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb71d7fa700 10 client.15234 ms_handle_connect on v2:172.21.15.111:6836/3946741228 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb739a52700 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55ebe0faabf0 0x55ebe0fa70b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6b2e062d88a4e36 server_cookie=6625b6b2d365d81e in_seq=0 out_seq=0 2023-12-12T09:57:58.897 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb71d7fa700 1 -- 172.21.15.42:0/533117741 <== mds.3 v2:172.21.15.111:6836/3946741228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55ebe0e50f80 con 0x55ebe0faabf0 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb71d7fa700 10 client.15234 handle_command_reply: tid=0 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 client.15234 shutdown 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 -- 172.21.15.42:0/533117741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 msgr2=0x55ebe0f4b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4b190 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb72c005900 tx=0x7fb72c007590 comp rx=0 tx=0).stop 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 -- 172.21.15.42:0/533117741 shutdown_connections 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55ebe0faabf0 0x55ebe0fa70b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ebe0f3bfe0 0x55ebe0f4b190 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb71d7fa700 0 client.15234 ms_handle_reset on v2:172.21.15.111:6836/3946741228 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ebe0f3a790 0x55ebe0f47ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 --2- 172.21.15.42:0/533117741 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ebe0f3b0f0 0x55ebe0f48220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.898 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 -- 172.21.15.42:0/533117741 >> 172.21.15.42:0/533117741 conn(0x55ebe0e65790 msgr2=0x55ebe0f58b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.899 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 -- 172.21.15.42:0/533117741 shutdown_connections 2023-12-12T09:57:58.899 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.888+0000 7fb747eafb80 1 -- 172.21.15.42:0/533117741 wait complete. 2023-12-12T09:57:58.899 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb747eafb80 20 client.15234 trim_cache size 0 max 16384 2023-12-12T09:57:58.899 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:57:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:57:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 3, 2023-12-12T09:57:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24482, 2023-12-12T09:57:58.899 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:57:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:57:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:57:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 53.601728356999999, 2023-12-12T09:57:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:57:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:57:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:57:58.900 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 95.951133421999998 2023-12-12T09:57:58.900 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:57:58.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 -- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb7200a1590 msgr2=0x7fb7200a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb7200a1590 0x7fb7200a3a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb730009bf0 tx=0x7fb730009260 comp rx=0 tx=0).stop 2023-12-12T09:57:58.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 -- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7341004b0 msgr2=0x7fb734196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:58.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7341004b0 0x7fb734196490 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb72400e970 tx=0x7fb72400ec80 comp rx=0 tx=0).stop 2023-12-12T09:57:58.901 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 -- 172.21.15.42:0/255343541 shutdown_connections 2023-12-12T09:57:58.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb7200a1590 0x7fb7200a3a50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb734102040 0x7fb73419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb7341004b0 0x7fb734196490 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 --2- 172.21.15.42:0/255343541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7341016a0 0x7fb7341969d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:58.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 -- 172.21.15.42:0/255343541 >> 172.21.15.42:0/255343541 conn(0x7fb7340fba20 msgr2=0x7fb7341049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:58.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 -- 172.21.15.42:0/255343541 shutdown_connections 2023-12-12T09:57:58.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:58.889+0000 7fb73bcb6700 1 -- 172.21.15.42:0/255343541 wait complete. 2023-12-12T09:57:58.903 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-12T09:57:59.030 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 -- 172.21.15.42:0/4273947178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41016a0 msgr2=0x7ff2b4101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.030 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 --2- 172.21.15.42:0/4273947178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41016a0 0x7ff2b4101b00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80099c0 tx=0x7ff2a8009cd0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.030 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 -- 172.21.15.42:0/4273947178 shutdown_connections 2023-12-12T09:57:59.030 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 --2- 172.21.15.42:0/4273947178 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2b4102040 0x7ff2b41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.031 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 --2- 172.21.15.42:0/4273947178 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b41004b0 0x7ff2b41008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.031 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 --2- 172.21.15.42:0/4273947178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41016a0 0x7ff2b4101b00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.031 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 -- 172.21.15.42:0/4273947178 >> 172.21.15.42:0/4273947178 conn(0x7ff2b40fba20 msgr2=0x7ff2b40fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.031 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 -- 172.21.15.42:0/4273947178 shutdown_connections 2023-12-12T09:57:59.031 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 -- 172.21.15.42:0/4273947178 wait complete. 2023-12-12T09:57:59.031 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.028+0000 7ff2b9346700 1 Processor -- start 2023-12-12T09:57:59.033 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b9346700 1 -- start start 2023-12-12T09:57:59.033 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b9346700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41004b0 0x7ff2b4196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.033 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b9346700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b41016a0 0x7ff2b41969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.033 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b9346700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2b4102040 0x7ff2b419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.033 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b9346700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419df90 con 0x7ff2b41004b0 2023-12-12T09:57:59.033 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b9346700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419e0d0 con 0x7ff2b4102040 2023-12-12T09:57:59.033 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b9346700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419e260 con 0x7ff2b41016a0 2023-12-12T09:57:59.034 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41004b0 0x7ff2b4196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.034 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41004b0 0x7ff2b4196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34748/0 (socket says 172.21.15.42:34748) 2023-12-12T09:57:59.034 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b2ffd700 1 -- 172.21.15.42:0/307440628 learned_addr learned my addr 172.21.15.42:0/307440628 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.034 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b41016a0 0x7ff2b41969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.034 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b2ffd700 1 -- 172.21.15.42:0/307440628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b41016a0 msgr2=0x7ff2b41969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.034 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b41016a0 0x7ff2b41969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.034 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b2ffd700 1 -- 172.21.15.42:0/307440628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2b4102040 msgr2=0x7ff2b419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.034 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2b4102040 0x7ff2b419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.035 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b2ffd700 1 -- 172.21.15.42:0/307440628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80095f0 con 0x7ff2b41004b0 2023-12-12T09:57:59.035 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.029+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b41016a0 0x7ff2b41969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:59.035 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.030+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41004b0 0x7ff2b4196460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400d980 tx=0x7ff2a400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.035 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.030+0000 7ff29bfff700 1 -- 172.21.15.42:0/307440628 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400de60 con 0x7ff2b41004b0 2023-12-12T09:57:59.035 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.030+0000 7ff29bfff700 1 -- 172.21.15.42:0/307440628 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a40044e0 con 0x7ff2b41004b0 2023-12-12T09:57:59.035 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.030+0000 7ff2b9346700 1 -- 172.21.15.42:0/307440628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b4069c30 con 0x7ff2b41004b0 2023-12-12T09:57:59.035 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.030+0000 7ff29bfff700 1 -- 172.21.15.42:0/307440628 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4003b10 con 0x7ff2b41004b0 2023-12-12T09:57:59.035 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.030+0000 7ff2b9346700 1 -- 172.21.15.42:0/307440628 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b406a190 con 0x7ff2b41004b0 2023-12-12T09:57:59.042 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.032+0000 7ff29bfff700 1 -- 172.21.15.42:0/307440628 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7ff2a4009b30 con 0x7ff2b41004b0 2023-12-12T09:57:59.042 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.032+0000 7ff29bfff700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff29c0a1590 0x7ff29c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.042 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.032+0000 7ff29bfff700 1 -- 172.21.15.42:0/307440628 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff2a40cce30 con 0x7ff2b41004b0 2023-12-12T09:57:59.042 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.033+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff29c0a1590 0x7ff29c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.033+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff29c0a1590 0x7ff29c0a3a50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80098e0 tx=0x7ff2a8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.033+0000 7ff2c553fb80 1 Processor -- start 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.033+0000 7ff2c553fb80 1 -- start start 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a321b00 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a31f420 con 0x55fc1a3c6380 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a31ad10 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.043 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34758/0 (socket says 172.21.15.42:34758) 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b27fc700 1 -- 172.21.15.42:0/3723172021 learned_addr learned my addr 172.21.15.42:0/3723172021 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b37fe700 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b27fc700 1 -- 172.21.15.42:0/3723172021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 msgr2=0x55fc1a3c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b27fc700 1 -- 172.21.15.42:0/3723172021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 msgr2=0x55fc1a3c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b27fc700 1 -- 172.21.15.42:0/3723172021 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc1a1975e0 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.034+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3c75f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800bbd0 tx=0x7ff2a8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6730a594b8cc8c84 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2997fa700 1 -- 172.21.15.42:0/3723172021 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8030070 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2997fa700 1 -- 172.21.15.42:0/3723172021 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a8028040 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2997fa700 1 -- 172.21.15.42:0/3723172021 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a801c040 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.044 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2c553fb80 1 -- 172.21.15.42:0/3723172021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 msgr2=0x55fc1a3c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3c75f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800bbd0 tx=0x7ff2a8009310 comp rx=0 tx=0).stop 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2c553fb80 1 -- 172.21.15.42:0/3723172021 shutdown_connections 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/3723172021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3c75f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2c553fb80 1 -- 172.21.15.42:0/3723172021 >> 172.21.15.42:0/3723172021 conn(0x55fc1a2f5790 msgr2=0x55fc1a3c2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2c553fb80 1 -- 172.21.15.42:0/3723172021 shutdown_connections 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.035+0000 7ff2c553fb80 1 -- 172.21.15.42:0/3723172021 wait complete. 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2c553fb80 1 -- start start 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2c553fb80 1 Processor -- start 2023-12-12T09:57:59.045 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6380 0x55fc1a3d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a2d4510 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a2d41b0 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a1998d0 con 0x55fc1a3c6380 2023-12-12T09:57:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2c553fb80 10 client.0 _ll_register_callbacks cb 0x55fc1a1ffa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:59.046 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2b37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2b37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34766/0 (socket says 172.21.15.42:34766) 2023-12-12T09:57:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2b37fe700 1 -- 172.21.15.42:0/2632816592 learned_addr learned my addr 172.21.15.42:0/2632816592 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6380 0x55fc1a3d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.036+0000 7ff2b37fe700 1 -- 172.21.15.42:0/2632816592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6380 msgr2=0x55fc1a3d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2b37fe700 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6380 0x55fc1a3d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2b37fe700 1 -- 172.21.15.42:0/2632816592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6ce0 msgr2=0x55fc1a3d0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2b37fe700 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.047 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2b37fe700 1 -- 172.21.15.42:0/2632816592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b406a190 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2b37fe700 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d76d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac007ec0 tx=0x7ff2ac0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85296e1eb1903c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff296ff5700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff296ff5700 1 -- 172.21.15.42:0/2632816592 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac00e050 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff296ff5700 1 -- 172.21.15.42:0/2632816592 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff2ac010e40 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2c553fb80 1 -- 172.21.15.42:0/2632816592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc1a3b9af0 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff296ff5700 1 -- 172.21.15.42:0/2632816592 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2ac015440 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2c553fb80 10 client.15246 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2c553fb80 1 -- 172.21.15.42:0/2632816592 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff2b4109240 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff296ff5700 1 -- 172.21.15.42:0/2632816592 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fc1a3b9af0 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff296ff5700 1 -- 172.21.15.42:0/2632816592 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff2ac016710 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2c553fb80 10 client.15246 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.037+0000 7ff2c553fb80 10 client.15246 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-12T09:57:59.048 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.038+0000 7ff2c553fb80 20 client.15246 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.038+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55fc1a3dbe70 0x55fc1a3da8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.038+0000 7ff2c553fb80 4 client.15246 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.038+0000 7ff2c553fb80 1 -- 172.21.15.42:0/2632816592 --> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc1a395ef0 con 0x55fc1a3dbe70 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.038+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55fc1a3dbe70 0x55fc1a3da8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.038+0000 7ff296ff5700 10 client.15246 ms_handle_connect on v2:172.21.15.173:6836/1667746348 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.038+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55fc1a3dbe70 0x55fc1a3da8d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21896e33051e9971 server_cookie=446ea6abe7473cdd in_seq=0 out_seq=0 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff296ff5700 1 -- 172.21.15.42:0/2632816592 <== mds.2 v2:172.21.15.173:6836/1667746348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fc1a395ef0 con 0x55fc1a3dbe70 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff296ff5700 10 client.15246 handle_command_reply: tid=0 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 client.15246 shutdown 2023-12-12T09:57:59.049 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 -- 172.21.15.42:0/2632816592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 msgr2=0x55fc1a3d76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d76d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac007ec0 tx=0x7ff2ac0104a0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 -- 172.21.15.42:0/2632816592 shutdown_connections 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55fc1a3dbe70 0x55fc1a3da8d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d76d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6380 0x55fc1a3d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff296ff5700 0 client.15246 ms_handle_reset on v2:172.21.15.173:6836/1667746348 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/2632816592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 -- 172.21.15.42:0/2632816592 >> 172.21.15.42:0/2632816592 conn(0x55fc1a2f5790 msgr2=0x55fc1a3c19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 -- 172.21.15.42:0/2632816592 shutdown_connections 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.039+0000 7ff2c553fb80 1 -- 172.21.15.42:0/2632816592 wait complete. 2023-12-12T09:57:59.050 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.040+0000 7ff2c553fb80 20 client.15246 trim_cache size 0 max 16384 2023-12-12T09:57:59.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.057+0000 7ff2c553fb80 1 Processor -- start 2023-12-12T09:57:59.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.057+0000 7ff2c553fb80 1 -- start start 2023-12-12T09:57:59.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a2bd870 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a3d9490 con 0x55fc1a3c6380 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a3d1050 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34772/0 (socket says 172.21.15.42:34772) 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b27fc700 1 -- 172.21.15.42:0/1599783865 learned_addr learned my addr 172.21.15.42:0/1599783865 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b37fe700 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b27fc700 1 -- 172.21.15.42:0/1599783865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 msgr2=0x55fc1a3d0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b27fc700 1 -- 172.21.15.42:0/1599783865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 msgr2=0x55fc1a3c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b27fc700 1 -- 172.21.15.42:0/1599783865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc1a143d90 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.066 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b37fe700 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d75f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800f740 tx=0x7ff2a800f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c0d0654263f6ee server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff299ffb700 1 -- 172.21.15.42:0/1599783865 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a801c030 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff299ffb700 1 -- 172.21.15.42:0/1599783865 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a800fe50 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.058+0000 7ff299ffb700 1 -- 172.21.15.42:0/1599783865 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8030040 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 -- 172.21.15.42:0/1599783865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 msgr2=0x55fc1a3d75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d75f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800f740 tx=0x7ff2a800f8f0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 -- 172.21.15.42:0/1599783865 shutdown_connections 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/1599783865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d75f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 -- 172.21.15.42:0/1599783865 >> 172.21.15.42:0/1599783865 conn(0x55fc1a2f5790 msgr2=0x55fc1a3cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 -- 172.21.15.42:0/1599783865 shutdown_connections 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 -- 172.21.15.42:0/1599783865 wait complete. 2023-12-12T09:57:59.067 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 -- start start 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.059+0000 7ff2c553fb80 1 Processor -- start 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3d4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2c553fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a3d7750 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a3d78c0 con 0x55fc1a3c6380 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2c553fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc1a431f30 con 0x55fc1a3c6ce0 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2c553fb80 10 client.0 _ll_register_callbacks cb 0x55fc1a1ffa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34780/0 (socket says 172.21.15.42:34780) 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b37fe700 1 -- 172.21.15.42:0/474817472 learned_addr learned my addr 172.21.15.42:0/474817472 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b27fc700 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3d4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b37fe700 1 -- 172.21.15.42:0/474817472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6ce0 msgr2=0x55fc1a3d6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b37fe700 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b37fe700 1 -- 172.21.15.42:0/474817472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 msgr2=0x55fc1a3d4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b37fe700 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3d4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b37fe700 1 -- 172.21.15.42:0/474817472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc1a1975e0 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3d4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2b37fe700 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d70b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac005c30 tx=0x7ff2ac009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35289d0f2643c7c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff296ff5700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff296ff5700 1 -- 172.21.15.42:0/474817472 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac00e040 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff296ff5700 1 -- 172.21.15.42:0/474817472 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff2ac010c40 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff2c553fb80 1 -- 172.21.15.42:0/474817472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2a8028040 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.060+0000 7ff296ff5700 1 -- 172.21.15.42:0/474817472 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2ac010e00 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff2c553fb80 10 client.15252 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff2c553fb80 1 -- 172.21.15.42:0/474817472 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55fc1a12cc10 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff296ff5700 1 -- 172.21.15.42:0/474817472 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff2a8028040 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff296ff5700 1 -- 172.21.15.42:0/474817472 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7ff2ac032060 con 0x55fc1a3c7bd0 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff2c553fb80 10 client.15252 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff2c553fb80 10 client.15252 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff2c553fb80 20 client.15252 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55fc1a4370c0 0x55fc1a433520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff2c553fb80 4 client.15252 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff2c553fb80 1 -- 172.21.15.42:0/474817472 --> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fc1a2e0f80 con 0x55fc1a4370c0 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.061+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55fc1a4370c0 0x55fc1a433520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff296ff5700 10 client.15252 ms_handle_connect on v2:172.21.15.173:6836/1667746348 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2b2ffd700 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55fc1a4370c0 0x55fc1a433520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=640c8c7d57864200 server_cookie=fb01879756d78be6 in_seq=0 out_seq=0 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff296ff5700 1 -- 172.21.15.42:0/474817472 <== mds.2 v2:172.21.15.173:6836/1667746348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55fc1a2e0f80 con 0x55fc1a4370c0 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff296ff5700 10 client.15252 handle_command_reply: tid=0 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 client.15252 shutdown 2023-12-12T09:57:59.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 -- 172.21.15.42:0/474817472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 msgr2=0x55fc1a3d70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d70b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac005c30 tx=0x7ff2ac009660 comp rx=0 tx=0).stop 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 -- 172.21.15.42:0/474817472 shutdown_connections 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55fc1a4370c0 0x55fc1a433520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55fc1a3c6380 0x55fc1a3d4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff296ff5700 0 client.15252 ms_handle_reset on v2:172.21.15.173:6836/1667746348 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fc1a3c7bd0 0x55fc1a3d70b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 --2- 172.21.15.42:0/474817472 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55fc1a3c6ce0 0x55fc1a3d6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 -- 172.21.15.42:0/474817472 >> 172.21.15.42:0/474817472 conn(0x55fc1a2f5790 msgr2=0x55fc1a3d00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 -- 172.21.15.42:0/474817472 shutdown_connections 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 1 -- 172.21.15.42:0/474817472 wait complete. 2023-12-12T09:57:59.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.062+0000 7ff2c553fb80 20 client.15252 trim_cache size 0 max 16384 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24487, 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 54.784960951000002, 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 98.282396415999997 2023-12-12T09:57:59.072 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:57:59.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 -- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff29c0a1590 msgr2=0x7ff29c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff29c0a1590 0x7ff29c0a3a50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80098e0 tx=0x7ff2a8019040 comp rx=0 tx=0).stop 2023-12-12T09:57:59.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 -- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41004b0 msgr2=0x7ff2b4196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41004b0 0x7ff2b4196460 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400d980 tx=0x7ff2a400dd40 comp rx=0 tx=0).stop 2023-12-12T09:57:59.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 -- 172.21.15.42:0/307440628 shutdown_connections 2023-12-12T09:57:59.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7ff29c0a1590 0x7ff29c0a3a50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2b4102040 0x7ff2b419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2b41004b0 0x7ff2b4196460 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 --2- 172.21.15.42:0/307440628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff2b41016a0 0x7ff2b41969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 -- 172.21.15.42:0/307440628 >> 172.21.15.42:0/307440628 conn(0x7ff2b40fba20 msgr2=0x7ff2b4104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 -- 172.21.15.42:0/307440628 shutdown_connections 2023-12-12T09:57:59.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.063+0000 7ff2b9346700 1 -- 172.21.15.42:0/307440628 wait complete. 2023-12-12T09:57:59.077 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-12T09:57:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.202+0000 7f7346198700 1 -- 172.21.15.42:0/1869480480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7340103190 msgr2=0x7f7340109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.202+0000 7f7346198700 1 --2- 172.21.15.42:0/1869480480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7340103190 0x7f7340109a40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7338009990 tx=0x7f7338009ca0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.202+0000 7f7346198700 1 -- 172.21.15.42:0/1869480480 shutdown_connections 2023-12-12T09:57:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.202+0000 7f7346198700 1 --2- 172.21.15.42:0/1869480480 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73400ff780 0x7f73400ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.202+0000 7f7346198700 1 --2- 172.21.15.42:0/1869480480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7340103190 0x7f7340109a40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.202+0000 7f7346198700 1 --2- 172.21.15.42:0/1869480480 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f73401027f0 0x7f7340102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.202+0000 7f7346198700 1 -- 172.21.15.42:0/1869480480 >> 172.21.15.42:0/1869480480 conn(0x7f73400fb090 msgr2=0x7f73400fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.202+0000 7f7346198700 1 -- 172.21.15.42:0/1869480480 shutdown_connections 2023-12-12T09:57:59.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.202+0000 7f7346198700 1 -- 172.21.15.42:0/1869480480 wait complete. 2023-12-12T09:57:59.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f7346198700 1 Processor -- start 2023-12-12T09:57:59.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f7346198700 1 -- start start 2023-12-12T09:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f7346198700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73400ff780 0x7f73401973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f7346198700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73401027f0 0x7f7340197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f7346198700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7340103190 0x7f734019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f7346198700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734019ef00 con 0x7f73400ff780 2023-12-12T09:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f7346198700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734019f040 con 0x7f73401027f0 2023-12-12T09:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f7346198700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734019f1d0 con 0x7f7340103190 2023-12-12T09:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73400ff780 0x7f73401973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73400ff780 0x7f73401973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34790/0 (socket says 172.21.15.42:34790) 2023-12-12T09:57:59.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f733f7fe700 1 -- 172.21.15.42:0/2444884617 learned_addr learned my addr 172.21.15.42:0/2444884617 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f733ffff700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7340103190 0x7f734019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.203+0000 7f733effd700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73401027f0 0x7f7340197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733f7fe700 1 -- 172.21.15.42:0/2444884617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7340103190 msgr2=0x7f734019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733f7fe700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7340103190 0x7f734019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733f7fe700 1 -- 172.21.15.42:0/2444884617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73401027f0 msgr2=0x7f7340197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733f7fe700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73401027f0 0x7f7340197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733f7fe700 1 -- 172.21.15.42:0/2444884617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73380095f0 con 0x7f73400ff780 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733effd700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73401027f0 0x7f7340197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733ffff700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7340103190 0x7f734019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733f7fe700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73400ff780 0x7f73401973d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f733000e970 tx=0x7f733000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733cff9700 1 -- 172.21.15.42:0/2444884617 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330009790 con 0x7f73400ff780 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733cff9700 1 -- 172.21.15.42:0/2444884617 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73300044e0 con 0x7f73400ff780 2023-12-12T09:57:59.218 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f7346198700 1 -- 172.21.15.42:0/2444884617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734019f4b0 con 0x7f73400ff780 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f733cff9700 1 -- 172.21.15.42:0/2444884617 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330003e30 con 0x7f73400ff780 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.204+0000 7f7346198700 1 -- 172.21.15.42:0/2444884617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734019f9c0 con 0x7f73400ff780 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.206+0000 7f733cff9700 1 -- 172.21.15.42:0/2444884617 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f7330003f90 con 0x7f73400ff780 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.207+0000 7f733cff9700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73280a1500 0x7f73280a39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.207+0000 7f733cff9700 1 -- 172.21.15.42:0/2444884617 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7330014070 con 0x7f73400ff780 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.207+0000 7f733effd700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73280a1500 0x7f73280a39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.207+0000 7f733effd700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73280a1500 0x7f73280a39c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7334009dd0 tx=0x7f7334009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f7352391b80 1 Processor -- start 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f7352391b80 1 -- start start 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f7352391b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fa380 0x5588cd6fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f7352391b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6face0 0x5588cd6fb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f7352391b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fbbd0 0x5588cd6fc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f7352391b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd655b00 con 0x5588cd6face0 2023-12-12T09:57:59.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f7352391b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd653420 con 0x5588cd6fbbd0 2023-12-12T09:57:59.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f7352391b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd64ed10 con 0x5588cd6fa380 2023-12-12T09:57:59.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6face0 0x5588cd6fb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6face0 0x5588cd6fb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34800/0 (socket says 172.21.15.42:34800) 2023-12-12T09:57:59.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733effd700 1 -- 172.21.15.42:0/4136238651 learned_addr learned my addr 172.21.15.42:0/4136238651 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733f7fe700 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fa380 0x5588cd6fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733ffff700 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fbbd0 0x5588cd6fc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733effd700 1 -- 172.21.15.42:0/4136238651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fa380 msgr2=0x5588cd6fa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733effd700 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fa380 0x5588cd6fa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733effd700 1 -- 172.21.15.42:0/4136238651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fbbd0 msgr2=0x5588cd6fc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733effd700 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fbbd0 0x5588cd6fc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733effd700 1 -- 172.21.15.42:0/4136238651 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588cd4cb5e0 con 0x5588cd6face0 2023-12-12T09:57:59.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733f7fe700 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fa380 0x5588cd6fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:59.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.208+0000 7f733effd700 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6face0 0x5588cd6fb5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7334016550 tx=0x7f7334012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02015b3d4652a7a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7325ffb700 1 -- 172.21.15.42:0/4136238651 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334021600 con 0x5588cd6face0 2023-12-12T09:57:59.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7325ffb700 1 -- 172.21.15.42:0/4136238651 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7334021c20 con 0x5588cd6face0 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7325ffb700 1 -- 172.21.15.42:0/4136238651 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334014cb0 con 0x5588cd6face0 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7352391b80 1 -- 172.21.15.42:0/4136238651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6face0 msgr2=0x5588cd6fb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7352391b80 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6face0 0x5588cd6fb5f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7334016550 tx=0x7f7334012000 comp rx=0 tx=0).stop 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7352391b80 1 -- 172.21.15.42:0/4136238651 shutdown_connections 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7352391b80 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fbbd0 0x5588cd6fc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7352391b80 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fa380 0x5588cd6fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7352391b80 1 --2- 172.21.15.42:0/4136238651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6face0 0x5588cd6fb5f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7352391b80 1 -- 172.21.15.42:0/4136238651 >> 172.21.15.42:0/4136238651 conn(0x5588cd629790 msgr2=0x5588cd6f6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7352391b80 1 -- 172.21.15.42:0/4136238651 shutdown_connections 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.209+0000 7f7352391b80 1 -- 172.21.15.42:0/4136238651 wait complete. 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f7352391b80 1 -- start start 2023-12-12T09:57:59.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f7352391b80 1 Processor -- start 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f7352391b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fa380 0x5588cd704190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f7352391b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6face0 0x5588cd7046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f7352391b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fbbd0 0x5588cd70b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f7352391b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd608510 con 0x5588cd6fbbd0 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f7352391b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd6081b0 con 0x5588cd6fa380 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f7352391b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd4cd8d0 con 0x5588cd6face0 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f7352391b80 10 client.0 _ll_register_callbacks cb 0x5588cd533a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f733ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fbbd0 0x5588cd70b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.210+0000 7f733ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fbbd0 0x5588cd70b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34808/0 (socket says 172.21.15.42:34808) 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f733ffff700 1 -- 172.21.15.42:0/494618673 learned_addr learned my addr 172.21.15.42:0/494618673 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f733f7fe700 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fa380 0x5588cd704190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f733effd700 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6face0 0x5588cd7046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f733ffff700 1 -- 172.21.15.42:0/494618673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6face0 msgr2=0x5588cd7046d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f733ffff700 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6face0 0x5588cd7046d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f733ffff700 1 -- 172.21.15.42:0/494618673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fa380 msgr2=0x5588cd704190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f733ffff700 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fa380 0x5588cd704190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f733ffff700 1 -- 172.21.15.42:0/494618673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7334007de0 con 0x5588cd6fbbd0 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f733ffff700 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fbbd0 0x5588cd70b710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f733800b680 tx=0x7f733800bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16652da33f9a3463 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f731effd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f731effd700 1 -- 172.21.15.42:0/494618673 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733800f8f0 con 0x5588cd6fbbd0 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f731effd700 1 -- 172.21.15.42:0/494618673 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f733800fa50 con 0x5588cd6fbbd0 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f7352391b80 1 -- 172.21.15.42:0/494618673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588cd6edaf0 con 0x5588cd6fbbd0 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f731effd700 1 -- 172.21.15.42:0/494618673 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f733800fc10 con 0x5588cd6fbbd0 2023-12-12T09:57:59.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f7352391b80 10 client.15264 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.211+0000 7f7352391b80 1 -- 172.21.15.42:0/494618673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588cd477d90 con 0x5588cd6fbbd0 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.212+0000 7f731effd700 1 -- 172.21.15.42:0/494618673 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588cd6edaf0 con 0x5588cd6fbbd0 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.212+0000 7f731effd700 1 -- 172.21.15.42:0/494618673 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f7338017480 con 0x5588cd6fbbd0 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.212+0000 7f7352391b80 10 client.15264 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.212+0000 7f7352391b80 10 client.15264 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.212+0000 7f7352391b80 20 client.15264 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.212+0000 7f7352391b80 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x5588cd70fe00 0x5588cd70e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.212+0000 7f7352391b80 4 client.15264 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.212+0000 7f7352391b80 1 -- 172.21.15.42:0/494618673 --> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588cd6c9ef0 con 0x5588cd70fe00 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.212+0000 7f733f7fe700 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x5588cd70fe00 0x5588cd70e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.213+0000 7f731effd700 10 client.15264 ms_handle_connect on v2:172.21.15.173:6838/2977372273 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.213+0000 7f733f7fe700 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x5588cd70fe00 0x5588cd70e8b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b77e6321b745d73d server_cookie=eecda4f3c2de0ff9 in_seq=0 out_seq=0 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.213+0000 7f731effd700 1 -- 172.21.15.42:0/494618673 <== mds.1 v2:172.21.15.173:6838/2977372273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588cd6c9ef0 con 0x5588cd70fe00 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.213+0000 7f731effd700 10 client.15264 handle_command_reply: tid=0 2023-12-12T09:57:59.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.213+0000 7f7352391b80 1 client.15264 shutdown 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 -- 172.21.15.42:0/494618673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fbbd0 msgr2=0x5588cd70b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fbbd0 0x5588cd70b710 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f733800b680 tx=0x7f733800bc50 comp rx=0 tx=0).stop 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 -- 172.21.15.42:0/494618673 shutdown_connections 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6fa380 0x5588cd704190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x5588cd70fe00 0x5588cd70e8b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fbbd0 0x5588cd70b710 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 --2- 172.21.15.42:0/494618673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6face0 0x5588cd7046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f731effd700 0 client.15264 ms_handle_reset on v2:172.21.15.173:6838/2977372273 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 -- 172.21.15.42:0/494618673 >> 172.21.15.42:0/494618673 conn(0x5588cd629790 msgr2=0x5588cd6f59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 -- 172.21.15.42:0/494618673 shutdown_connections 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 1 -- 172.21.15.42:0/494618673 wait complete. 2023-12-12T09:57:59.226 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.214+0000 7f7352391b80 20 client.15264 trim_cache size 0 max 16384 2023-12-12T09:57:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f7352391b80 1 Processor -- start 2023-12-12T09:57:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f7352391b80 1 -- start start 2023-12-12T09:57:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f7352391b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd7036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.240 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f7352391b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 0x5588cd70b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f7352391b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 0x5588cd6fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f7352391b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd64ae00 con 0x5588cd6fa380 2023-12-12T09:57:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f7352391b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd5f3b50 con 0x5588cd6face0 2023-12-12T09:57:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f7352391b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd5f1830 con 0x5588cd6fbbd0 2023-12-12T09:57:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd7036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd7036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34812/0 (socket says 172.21.15.42:34812) 2023-12-12T09:57:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.232+0000 7f733f7fe700 1 -- 172.21.15.42:0/3281229975 learned_addr learned my addr 172.21.15.42:0/3281229975 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f733ffff700 1 --2- 172.21.15.42:0/3281229975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 0x5588cd6fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f733f7fe700 1 -- 172.21.15.42:0/3281229975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 msgr2=0x5588cd6fbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f733f7fe700 1 --2- 172.21.15.42:0/3281229975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 0x5588cd6fbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f733f7fe700 1 -- 172.21.15.42:0/3281229975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 msgr2=0x5588cd70b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f733effd700 1 --2- 172.21.15.42:0/3281229975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 0x5588cd70b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f733f7fe700 1 --2- 172.21.15.42:0/3281229975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 0x5588cd70b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f733f7fe700 1 -- 172.21.15.42:0/3281229975 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588cd460c10 con 0x5588cd6fa380 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f733f7fe700 1 --2- 172.21.15.42:0/3281229975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd7036a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f733009a330 tx=0x7f733009a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3271f6c166370f7a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f73267fc700 1 -- 172.21.15.42:0/3281229975 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733009e4f0 con 0x5588cd6fa380 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f73267fc700 1 -- 172.21.15.42:0/3281229975 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73300a2730 con 0x5588cd6fa380 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.233+0000 7f73267fc700 1 -- 172.21.15.42:0/3281229975 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330004240 con 0x5588cd6fa380 2023-12-12T09:57:59.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 -- 172.21.15.42:0/3281229975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 msgr2=0x5588cd7036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 --2- 172.21.15.42:0/3281229975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd7036a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f733009a330 tx=0x7f733009a1d0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 -- 172.21.15.42:0/3281229975 shutdown_connections 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 --2- 172.21.15.42:0/3281229975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 0x5588cd6fbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 --2- 172.21.15.42:0/3281229975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd7036a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 --2- 172.21.15.42:0/3281229975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 0x5588cd70b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 -- 172.21.15.42:0/3281229975 >> 172.21.15.42:0/3281229975 conn(0x5588cd629790 msgr2=0x5588cd702190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 -- 172.21.15.42:0/3281229975 shutdown_connections 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 -- 172.21.15.42:0/3281229975 wait complete. 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 -- start start 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 Processor -- start 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd70b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 0x5588cd70b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 0x5588cd707c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd7083d0 con 0x5588cd6fa380 2023-12-12T09:57:59.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd708540 con 0x5588cd6face0 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.234+0000 7f7352391b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588cd708700 con 0x5588cd6fbbd0 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f7352391b80 10 client.0 _ll_register_callbacks cb 0x5588cd533a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd70b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd70b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34818/0 (socket says 172.21.15.42:34818) 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733f7fe700 1 -- 172.21.15.42:0/3794772627 learned_addr learned my addr 172.21.15.42:0/3794772627 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733ffff700 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 0x5588cd707c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733f7fe700 1 -- 172.21.15.42:0/3794772627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 msgr2=0x5588cd707c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733effd700 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 0x5588cd70b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733f7fe700 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 0x5588cd707c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733f7fe700 1 -- 172.21.15.42:0/3794772627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 msgr2=0x5588cd70b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733f7fe700 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 0x5588cd70b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733f7fe700 1 -- 172.21.15.42:0/3794772627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734019f9c0 con 0x5588cd6fa380 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733ffff700 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 0x5588cd707c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:57:59.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f733f7fe700 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd70b0b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f73300cac10 tx=0x7f7330098590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec50a7c995414e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f731effd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f731effd700 1 -- 172.21.15.42:0/3794772627 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73300cade0 con 0x5588cd6fa380 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f731effd700 1 -- 172.21.15.42:0/3794772627 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7330004240 con 0x5588cd6fa380 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f7352391b80 1 -- 172.21.15.42:0/3794772627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7334021c20 con 0x5588cd6fa380 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f731effd700 1 -- 172.21.15.42:0/3794772627 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73300cd280 con 0x5588cd6fa380 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f7352391b80 10 client.15270 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f7352391b80 1 -- 172.21.15.42:0/3794772627 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5588cd49e200 con 0x5588cd6fa380 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.235+0000 7f731effd700 1 -- 172.21.15.42:0/3794772627 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7334021c20 con 0x5588cd6fa380 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f731effd700 1 -- 172.21.15.42:0/3794772627 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f733007a030 con 0x5588cd6fa380 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f7352391b80 10 client.15270 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f7352391b80 10 client.15270 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f7352391b80 20 client.15270 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f7352391b80 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x5588cd76ab40 0x5588cd766fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f7352391b80 4 client.15270 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:57:59.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f7352391b80 1 -- 172.21.15.42:0/3794772627 --> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5588cd614f80 con 0x5588cd76ab40 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f733effd700 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x5588cd76ab40 0x5588cd766fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f731effd700 10 client.15270 ms_handle_connect on v2:172.21.15.173:6838/2977372273 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.236+0000 7f733effd700 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x5588cd76ab40 0x5588cd766fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7fc4f42937e19bcd server_cookie=7c3ec629898e4c2f in_seq=0 out_seq=0 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f731effd700 1 -- 172.21.15.42:0/3794772627 <== mds.1 v2:172.21.15.173:6838/2977372273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5588cd614f80 con 0x5588cd76ab40 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f731effd700 10 client.15270 handle_command_reply: tid=0 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 client.15270 shutdown 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 -- 172.21.15.42:0/3794772627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 msgr2=0x5588cd70b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd70b0b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f73300cac10 tx=0x7f7330098590 comp rx=0 tx=0).stop 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 -- 172.21.15.42:0/3794772627 shutdown_connections 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x5588cd76ab40 0x5588cd766fe0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5588cd6fbbd0 0x5588cd707c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588cd6fa380 0x5588cd70b0b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 --2- 172.21.15.42:0/3794772627 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5588cd6face0 0x5588cd70b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f731effd700 0 client.15270 ms_handle_reset on v2:172.21.15.173:6838/2977372273 2023-12-12T09:57:59.246 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 -- 172.21.15.42:0/3794772627 >> 172.21.15.42:0/3794772627 conn(0x5588cd629790 msgr2=0x5588cd6f6920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 -- 172.21.15.42:0/3794772627 shutdown_connections 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 1 -- 172.21.15.42:0/3794772627 wait complete. 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7352391b80 20 client.15270 trim_cache size 0 max 16384 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24508, 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:57:59.247 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 55.970515200000001, 2023-12-12T09:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 94.024762639000002 2023-12-12T09:57:59.248 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7346198700 1 -- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73280a1500 msgr2=0x7f73280a39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7346198700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73280a1500 0x7f73280a39c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7334009dd0 tx=0x7f7334009040 comp rx=0 tx=0).stop 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7346198700 1 -- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73400ff780 msgr2=0x7f73401973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.237+0000 7f7346198700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73400ff780 0x7f73401973d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f733000e970 tx=0x7f733000ed30 comp rx=0 tx=0).stop 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.238+0000 7f7346198700 1 -- 172.21.15.42:0/2444884617 shutdown_connections 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.238+0000 7f7346198700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f73280a1500 0x7f73280a39c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.238+0000 7f7346198700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7340103190 0x7f734019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.238+0000 7f7346198700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73400ff780 0x7f73401973d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.238+0000 7f7346198700 1 --2- 172.21.15.42:0/2444884617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f73401027f0 0x7f7340197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.238+0000 7f7346198700 1 -- 172.21.15.42:0/2444884617 >> 172.21.15.42:0/2444884617 conn(0x7f73400fb090 msgr2=0x7f7340105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.238+0000 7f7346198700 1 -- 172.21.15.42:0/2444884617 shutdown_connections 2023-12-12T09:57:59.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.238+0000 7f7346198700 1 -- 172.21.15.42:0/2444884617 wait complete. 2023-12-12T09:57:59.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-12T09:57:59.381 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 -- 172.21.15.42:0/3191432180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 msgr2=0x7fcac41096d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.381 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 --2- 172.21.15.42:0/3191432180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 0x7fcac41096d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fcabc00b0a0 tx=0x7fcabc00b3b0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.381 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 -- 172.21.15.42:0/3191432180 shutdown_connections 2023-12-12T09:57:59.381 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 --2- 172.21.15.42:0/3191432180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 0x7fcac41096d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 --2- 172.21.15.42:0/3191432180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac4102000 0x7fcac41043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 --2- 172.21.15.42:0/3191432180 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac4104930 0x7fcac4106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 -- 172.21.15.42:0/3191432180 >> 172.21.15.42:0/3191432180 conn(0x7fcac40fb9f0 msgr2=0x7fcac40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 -- 172.21.15.42:0/3191432180 shutdown_connections 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 -- 172.21.15.42:0/3191432180 wait complete. 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 Processor -- start 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.377+0000 7fcacbaff700 1 -- start start 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcacbaff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac4102000 0x7fcac4101920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcacbaff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac4104930 0x7fcac40fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.382 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcacbaff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 0x7fcac41004f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcacbaff700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4100c90 con 0x7fcac4107260 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcacbaff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac4100e00 con 0x7fcac4102000 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcacbaff700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac41a15f0 con 0x7fcac4104930 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcaca09c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 0x7fcac41004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcaca09c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 0x7fcac41004f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34836/0 (socket says 172.21.15.42:34836) 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcaca09c700 1 -- 172.21.15.42:0/164162364 learned_addr learned my addr 172.21.15.42:0/164162364 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcac989b700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac4102000 0x7fcac4101920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcac909a700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac4104930 0x7fcac40fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcaca09c700 1 -- 172.21.15.42:0/164162364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac4104930 msgr2=0x7fcac40fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcaca09c700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac4104930 0x7fcac40fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcaca09c700 1 -- 172.21.15.42:0/164162364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac4102000 msgr2=0x7fcac4101920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcaca09c700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac4102000 0x7fcac4101920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcaca09c700 1 -- 172.21.15.42:0/164162364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc009d00 con 0x7fcac4107260 2023-12-12T09:57:59.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcac909a700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac4104930 0x7fcac40fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:59.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcaca09c700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 0x7fcac41004f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcabc004d00 tx=0x7fcabc004de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcabaffd700 1 -- 172.21.15.42:0/164162364 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc00e040 con 0x7fcac4107260 2023-12-12T09:57:59.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcabaffd700 1 -- 172.21.15.42:0/164162364 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcabc009440 con 0x7fcac4107260 2023-12-12T09:57:59.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcacbaff700 1 -- 172.21.15.42:0/164162364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac41a1820 con 0x7fcac4107260 2023-12-12T09:57:59.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcabaffd700 1 -- 172.21.15.42:0/164162364 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc01ba80 con 0x7fcac4107260 2023-12-12T09:57:59.385 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.378+0000 7fcacbaff700 1 -- 172.21.15.42:0/164162364 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac41a1ca0 con 0x7fcac4107260 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.381+0000 7fcabaffd700 1 -- 172.21.15.42:0/164162364 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fcabc01bbe0 con 0x7fcac4107260 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.381+0000 7fcabaffd700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcab00a1590 0x7fcab00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.381+0000 7fcac989b700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcab00a1590 0x7fcab00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.381+0000 7fcabaffd700 1 -- 172.21.15.42:0/164162364 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcabc0cecd0 con 0x7fcac4107260 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcac989b700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcab00a1590 0x7fcab00a3a50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcac0005fd0 tx=0x7fcac000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcad7cf8b80 1 Processor -- start 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcad7cf8b80 1 -- start start 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a79a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79ace0 0x56047a79b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79bbd0 0x56047a79c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a6f5b00 con 0x56047a79bbd0 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a6f3420 con 0x56047a79a380 2023-12-12T09:57:59.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a6eed10 con 0x56047a79ace0 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.382+0000 7fcaca09c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79bbd0 0x56047a79c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaca09c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79bbd0 0x56047a79c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34844/0 (socket says 172.21.15.42:34844) 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaca09c700 1 -- 172.21.15.42:0/1519319949 learned_addr learned my addr 172.21.15.42:0/1519319949 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcac989b700 1 --2- 172.21.15.42:0/1519319949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a79a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcac909a700 1 --2- 172.21.15.42:0/1519319949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79ace0 0x56047a79b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaca09c700 1 -- 172.21.15.42:0/1519319949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79ace0 msgr2=0x56047a79b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaca09c700 1 --2- 172.21.15.42:0/1519319949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79ace0 0x56047a79b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaca09c700 1 -- 172.21.15.42:0/1519319949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 msgr2=0x56047a79a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaca09c700 1 --2- 172.21.15.42:0/1519319949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a79a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaca09c700 1 -- 172.21.15.42:0/1519319949 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56047a56b5e0 con 0x56047a79bbd0 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaca09c700 1 --2- 172.21.15.42:0/1519319949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79bbd0 0x56047a79c050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0a39e0 tx=0x7fcabc09ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eff15f8adfad8770 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaaffff700 1 -- 172.21.15.42:0/1519319949 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc09c310 con 0x56047a79bbd0 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaaffff700 1 -- 172.21.15.42:0/1519319949 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcabc09bc10 con 0x56047a79bbd0 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.383+0000 7fcaaffff700 1 -- 172.21.15.42:0/1519319949 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcabc09c070 con 0x56047a79bbd0 2023-12-12T09:57:59.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1519319949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79bbd0 msgr2=0x56047a79c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/1519319949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79bbd0 0x56047a79c050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcabc0a39e0 tx=0x7fcabc09ba60 comp rx=0 tx=0).stop 2023-12-12T09:57:59.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1519319949 shutdown_connections 2023-12-12T09:57:59.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/1519319949 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a79a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/1519319949 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79bbd0 0x56047a79c050 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/1519319949 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79ace0 0x56047a79b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1519319949 >> 172.21.15.42:0/1519319949 conn(0x56047a6c9790 msgr2=0x56047a796d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1519319949 shutdown_connections 2023-12-12T09:57:59.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1519319949 wait complete. 2023-12-12T09:57:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 -- start start 2023-12-12T09:57:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.384+0000 7fcad7cf8b80 1 Processor -- start 2023-12-12T09:57:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79a380 0x56047a7a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79ace0 0x56047a7a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a7ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a6a8510 con 0x56047a79a380 2023-12-12T09:57:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a6a81b0 con 0x56047a79ace0 2023-12-12T09:57:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a56d8d0 con 0x56047a79bbd0 2023-12-12T09:57:59.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcad7cf8b80 10 client.0 _ll_register_callbacks cb 0x56047a5d3a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac989b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79a380 0x56047a7a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac989b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79a380 0x56047a7a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34846/0 (socket says 172.21.15.42:34846) 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac989b700 1 -- 172.21.15.42:0/6468094 learned_addr learned my addr 172.21.15.42:0/6468094 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac909a700 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79ace0 0x56047a7a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcaca09c700 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a7ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac989b700 1 -- 172.21.15.42:0/6468094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 msgr2=0x56047a7ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac989b700 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a7ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac989b700 1 -- 172.21.15.42:0/6468094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79ace0 msgr2=0x56047a7a44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac989b700 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79ace0 0x56047a7a44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac989b700 1 -- 172.21.15.42:0/6468094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac41a1ca0 con 0x56047a79a380 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.385+0000 7fcac909a700 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79ace0 0x56047a7a44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:57:59.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcac989b700 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79a380 0x56047a7a3fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcac00109e0 tx=0x7fcac0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecfc3ab59321294 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcaad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcaad7fa700 1 -- 172.21.15.42:0/6468094 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac0004a30 con 0x56047a79a380 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcaad7fa700 1 -- 172.21.15.42:0/6468094 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcac0004b90 con 0x56047a79a380 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/6468094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56047a78daf0 con 0x56047a79a380 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcaad7fa700 1 -- 172.21.15.42:0/6468094 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcac0004d50 con 0x56047a79a380 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcad7cf8b80 10 client.15282 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/6468094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56047a517d90 con 0x56047a79a380 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcaad7fa700 1 -- 172.21.15.42:0/6468094 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56047a78daf0 con 0x56047a79a380 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcaad7fa700 1 -- 172.21.15.42:0/6468094 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcac001ea20 con 0x56047a79a380 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcad7cf8b80 10 client.15282 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcad7cf8b80 10 client.15282 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:57:59.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcad7cf8b80 20 client.15282 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56047a7afc80 0x56047a7ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcad7cf8b80 4 client.15282 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.386+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/6468094 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56047a769ef0 con 0x56047a7afc80 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.387+0000 7fcac909a700 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56047a7afc80 0x56047a7ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.387+0000 7fcaad7fa700 10 client.15282 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.387+0000 7fcac909a700 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56047a7afc80 0x56047a7ae6e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90468a5acd32eed7 server_cookie=ca4414874e3263fc in_seq=0 out_seq=0 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcaad7fa700 1 -- 172.21.15.42:0/6468094 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56047a769ef0 con 0x56047a7afc80 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcaad7fa700 10 client.15282 handle_command_reply: tid=0 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 client.15282 shutdown 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/6468094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79a380 msgr2=0x56047a7a3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79a380 0x56047a7a3fb0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcac00109e0 tx=0x7fcac0004750 comp rx=0 tx=0).stop 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/6468094 shutdown_connections 2023-12-12T09:57:59.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a7ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79a380 0x56047a7a3fb0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56047a7afc80 0x56047a7ae6e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/6468094 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79ace0 0x56047a7a44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/6468094 >> 172.21.15.42:0/6468094 conn(0x56047a6c9790 msgr2=0x56047a7959f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcaad7fa700 0 client.15282 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:57:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/6468094 shutdown_connections 2023-12-12T09:57:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/6468094 wait complete. 2023-12-12T09:57:59.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.388+0000 7fcad7cf8b80 20 client.15282 trim_cache size 0 max 16384 2023-12-12T09:57:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcad7cf8b80 1 Processor -- start 2023-12-12T09:57:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcad7cf8b80 1 -- start start 2023-12-12T09:57:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a79bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.413 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a796920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a691830 con 0x56047a79ace0 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a799f20 con 0x56047a79a380 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a7ac660 con 0x56047a79bbd0 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac909a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac909a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34854/0 (socket says 172.21.15.42:34854) 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac909a700 1 -- 172.21.15.42:0/1588380103 learned_addr learned my addr 172.21.15.42:0/1588380103 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac989b700 1 --2- 172.21.15.42:0/1588380103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a79bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcaca09c700 1 --2- 172.21.15.42:0/1588380103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a796920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac909a700 1 -- 172.21.15.42:0/1588380103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 msgr2=0x56047a796920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac909a700 1 --2- 172.21.15.42:0/1588380103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a796920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac909a700 1 -- 172.21.15.42:0/1588380103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 msgr2=0x56047a79bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac909a700 1 --2- 172.21.15.42:0/1588380103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a79bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.414 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac909a700 1 -- 172.21.15.42:0/1588380103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56047a500c10 con 0x56047a79ace0 2023-12-12T09:57:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.407+0000 7fcac909a700 1 --2- 172.21.15.42:0/1588380103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7ab4a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x56047a7aba90 tx=0x7fcab4012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f382faa06bcb17 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcab8d82700 1 -- 172.21.15.42:0/1588380103 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4015040 con 0x56047a79ace0 2023-12-12T09:57:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcab8d82700 1 -- 172.21.15.42:0/1588380103 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcab401b8d0 con 0x56047a79ace0 2023-12-12T09:57:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcab8d82700 1 -- 172.21.15.42:0/1588380103 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4010de0 con 0x56047a79ace0 2023-12-12T09:57:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1588380103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 msgr2=0x56047a7ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/1588380103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7ab4a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x56047a7aba90 tx=0x7fcab4012ae0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1588380103 shutdown_connections 2023-12-12T09:57:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/1588380103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a79bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.415 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/1588380103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a796920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/1588380103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7ab4a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1588380103 >> 172.21.15.42:0/1588380103 conn(0x56047a6c9790 msgr2=0x56047a79eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1588380103 shutdown_connections 2023-12-12T09:57:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.408+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/1588380103 wait complete. 2023-12-12T09:57:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcad7cf8b80 1 -- start start 2023-12-12T09:57:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcad7cf8b80 1 Processor -- start 2023-12-12T09:57:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a7a75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcad7cf8b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a7aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a7aaff0 con 0x56047a79ace0 2023-12-12T09:57:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a7ab160 con 0x56047a79a380 2023-12-12T09:57:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcad7cf8b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047a7ab320 con 0x56047a79bbd0 2023-12-12T09:57:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcad7cf8b80 10 client.0 _ll_register_callbacks cb 0x56047a7ac0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:57:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcac909a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcac909a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7a7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34862/0 (socket says 172.21.15.42:34862) 2023-12-12T09:57:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcac909a700 1 -- 172.21.15.42:0/954037317 learned_addr learned my addr 172.21.15.42:0/954037317 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:57:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcaca09c700 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a7aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcac989b700 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a7a75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcac909a700 1 -- 172.21.15.42:0/954037317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 msgr2=0x56047a7aaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.409+0000 7fcac909a700 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a7aaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcac909a700 1 -- 172.21.15.42:0/954037317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 msgr2=0x56047a7a75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:57:59.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcac909a700 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a7a75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcac909a700 1 -- 172.21.15.42:0/954037317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab4012530 con 0x56047a79ace0 2023-12-12T09:57:59.418 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcac989b700 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a7a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:57:59.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcac909a700 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7a7b30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcab4005e50 tx=0x7fcab4010780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43ef2b7906979be server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:57:59.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcaad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:57:59.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcaad7fa700 1 -- 172.21.15.42:0/954037317 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4015070 con 0x56047a79ace0 2023-12-12T09:57:59.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcaad7fa700 1 -- 172.21.15.42:0/954037317 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcab4010b60 con 0x56047a79ace0 2023-12-12T09:57:59.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/954037317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcabc09bc10 con 0x56047a79ace0 2023-12-12T09:57:59.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcaad7fa700 1 -- 172.21.15.42:0/954037317 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab4010d20 con 0x56047a79ace0 2023-12-12T09:57:59.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcad7cf8b80 10 client.15288 fetch_fsmap learned FSMap version 27 2023-12-12T09:57:59.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/954037317 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56047a53e200 con 0x56047a79ace0 2023-12-12T09:57:59.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcaad7fa700 1 -- 172.21.15.42:0/954037317 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcabc09bc10 con 0x56047a79ace0 2023-12-12T09:57:59.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.410+0000 7fcaad7fa700 1 -- 172.21.15.42:0/954037317 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fcab402ebd0 con 0x56047a79ace0 2023-12-12T09:57:59.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.411+0000 7fcad7cf8b80 10 client.15288 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:57:59.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.411+0000 7fcad7cf8b80 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:57:59.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.411+0000 7fcad7cf8b80 20 client.15288 populate_metadata read hostname 'smithi042' 2023-12-12T09:57:59.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.411+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56047a80a930 0x56047a806d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:57:59.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.411+0000 7fcad7cf8b80 4 client.15288 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:57:59.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.411+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/954037317 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56047a6b4f80 con 0x56047a80a930 2023-12-12T09:57:59.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.411+0000 7fcac989b700 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56047a80a930 0x56047a806d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:57:59.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.411+0000 7fcac989b700 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56047a80a930 0x56047a806d90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67dfc1c706d88b51 server_cookie=ba6cc8888e129d5f in_seq=0 out_seq=0 2023-12-12T09:57:59.421 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.411+0000 7fcaad7fa700 10 client.15288 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:57:59.424 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcaad7fa700 1 -- 172.21.15.42:0/954037317 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x56047a6b4f80 con 0x56047a80a930 2023-12-12T09:57:59.424 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcaad7fa700 10 client.15288 handle_command_reply: tid=0 2023-12-12T09:57:59.424 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 client.15288 shutdown 2023-12-12T09:57:59.424 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/954037317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 msgr2=0x56047a7a7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7a7b30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcab4005e50 tx=0x7fcab4010780 comp rx=0 tx=0).stop 2023-12-12T09:57:59.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/954037317 shutdown_connections 2023-12-12T09:57:59.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56047a80a930 0x56047a806d90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56047a79a380 0x56047a7a75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcaad7fa700 0 client.15288 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:57:59.426 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56047a79bbd0 0x56047a7aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.426 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 --2- 172.21.15.42:0/954037317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047a79ace0 0x56047a7a7b30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.426 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/954037317 >> 172.21.15.42:0/954037317 conn(0x56047a6c9790 msgr2=0x56047a796920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.426 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/954037317 shutdown_connections 2023-12-12T09:57:59.426 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 1 -- 172.21.15.42:0/954037317 wait complete. 2023-12-12T09:57:59.427 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcad7cf8b80 20 client.15288 trim_cache size 0 max 16384 2023-12-12T09:57:59.427 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:57:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:57:59.427 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2023-12-12T09:57:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24524, 2023-12-12T09:57:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:57:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:57:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:57:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 80.951866979000002, 2023-12-12T09:57:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:57:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:57:59.428 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:57:59.429 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 87.031027154 2023-12-12T09:57:59.429 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:57:59.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcacbaff700 1 -- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcab00a1590 msgr2=0x7fcab00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcacbaff700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcab00a1590 0x7fcab00a3a50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcac0005fd0 tx=0x7fcac000a560 comp rx=0 tx=0).stop 2023-12-12T09:57:59.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcacbaff700 1 -- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 msgr2=0x7fcac41004f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:57:59.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcacbaff700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 0x7fcac41004f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fcabc004d00 tx=0x7fcabc004de0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.412+0000 7fcacbaff700 1 -- 172.21.15.42:0/164162364 shutdown_connections 2023-12-12T09:57:59.431 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.413+0000 7fcacbaff700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fcab00a1590 0x7fcab00a3a50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.413+0000 7fcacbaff700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac4102000 0x7fcac4101920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.413+0000 7fcacbaff700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcac4107260 0x7fcac41004f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.413+0000 7fcacbaff700 1 --2- 172.21.15.42:0/164162364 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcac4104930 0x7fcac40fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:57:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.413+0000 7fcacbaff700 1 -- 172.21.15.42:0/164162364 >> 172.21.15.42:0/164162364 conn(0x7fcac40fb9f0 msgr2=0x7fcac41029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:57:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.413+0000 7fcacbaff700 1 -- 172.21.15.42:0/164162364 shutdown_connections 2023-12-12T09:57:59.432 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:57:59.413+0000 7fcacbaff700 1 -- 172.21.15.42:0/164162364 wait complete. 2023-12-12T09:57:59.433 INFO:tasks.cephadm:Setup complete, yielding 2023-12-12T09:57:59.433 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-12T09:57:59.446 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T09:57:59.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- bash -c 'ceph orch status' 2023-12-12T09:57:59.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:59 smithi111 ceph-mon[102565]: pgmap v304: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:57:59.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:57:59 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2931342820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:59.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:59 smithi173 ceph-mon[103308]: pgmap v304: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:57:59.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:57:59 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2931342820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:57:59.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:59 smithi042 ceph-mon[98894]: pgmap v304: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:57:59.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:57:59 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2931342820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:58:00.047 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:58:01.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 -- 172.21.15.42:0/3371179184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c101720 msgr2=0x7f8e5c101b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:01.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 --2- 172.21.15.42:0/3371179184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c101720 0x7f8e5c101b80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8e500099c0 tx=0x7f8e50009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 -- 172.21.15.42:0/3371179184 shutdown_connections 2023-12-12T09:58:01.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 --2- 172.21.15.42:0/3371179184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e5c1020c0 0x7f8e5c108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 --2- 172.21.15.42:0/3371179184 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c100520 0x7f8e5c100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 --2- 172.21.15.42:0/3371179184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c101720 0x7f8e5c101b80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 -- 172.21.15.42:0/3371179184 >> 172.21.15.42:0/3371179184 conn(0x7f8e5c0fbae0 msgr2=0x7f8e5c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:01.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 -- 172.21.15.42:0/3371179184 shutdown_connections 2023-12-12T09:58:01.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 -- 172.21.15.42:0/3371179184 wait complete. 2023-12-12T09:58:01.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.229+0000 7f8e618d5700 1 Processor -- start 2023-12-12T09:58:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e618d5700 1 -- start start 2023-12-12T09:58:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e618d5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c100520 0x7f8e5c199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e618d5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e5c101720 0x7f8e5c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e618d5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c1020c0 0x7f8e5c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e618d5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c1a1260 con 0x7f8e5c100520 2023-12-12T09:58:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e618d5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c1a13a0 con 0x7f8e5c101720 2023-12-12T09:58:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e618d5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c1a1530 con 0x7f8e5c1020c0 2023-12-12T09:58:01.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c100520 0x7f8e5c199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c100520 0x7f8e5c199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34870/0 (socket says 172.21.15.42:34870) 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5affd700 1 -- 172.21.15.42:0/2511330465 learned_addr learned my addr 172.21.15.42:0/2511330465 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5a7fc700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e5c101720 0x7f8e5c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5affd700 1 -- 172.21.15.42:0/2511330465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c1020c0 msgr2=0x7f8e5c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5affd700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c1020c0 0x7f8e5c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5affd700 1 -- 172.21.15.42:0/2511330465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e5c101720 msgr2=0x7f8e5c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5affd700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e5c101720 0x7f8e5c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5affd700 1 -- 172.21.15.42:0/2511330465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e500095f0 con 0x7f8e5c100520 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.230+0000 7f8e5affd700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c100520 0x7f8e5c199720 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00d9b0 tx=0x7f8e4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.231+0000 7f8e608d3700 1 -- 172.21.15.42:0/2511330465 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00b950 con 0x7f8e5c100520 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.231+0000 7f8e608d3700 1 -- 172.21.15.42:0/2511330465 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e4c007620 con 0x7f8e5c100520 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.231+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c1a1810 con 0x7f8e5c100520 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.231+0000 7f8e608d3700 1 -- 172.21.15.42:0/2511330465 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00f3c0 con 0x7f8e5c100520 2023-12-12T09:58:01.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.231+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c1a1d60 con 0x7f8e5c100520 2023-12-12T09:58:01.236 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.232+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c193350 con 0x7f8e5c100520 2023-12-12T09:58:01.236 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.233+0000 7f8e608d3700 1 -- 172.21.15.42:0/2511330465 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f8e4c010640 con 0x7f8e5c100520 2023-12-12T09:58:01.236 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.233+0000 7f8e608d3700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8e440a1570 0x7f8e440a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:01.237 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.234+0000 7f8e5a7fc700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8e440a1570 0x7f8e440a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:01.237 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.234+0000 7f8e608d3700 1 -- 172.21.15.42:0/2511330465 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8e4c0cd880 con 0x7f8e5c100520 2023-12-12T09:58:01.237 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.234+0000 7f8e5a7fc700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8e440a1570 0x7f8e440a3a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009990 tx=0x7f8e50005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:01.239 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.237+0000 7f8e608d3700 1 -- 172.21.15.42:0/2511330465 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8e4c096a10 con 0x7f8e5c100520 2023-12-12T09:58:01.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.455+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e5c02d3b0 con 0x7f8e440a1570 2023-12-12T09:58:01.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.456+0000 7f8e608d3700 1 -- 172.21.15.42:0/2511330465 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f8e5c02d3b0 con 0x7f8e440a1570 2023-12-12T09:58:01.458 INFO:teuthology.orchestra.run.smithi042.stdout:Backend: cephadm 2023-12-12T09:58:01.459 INFO:teuthology.orchestra.run.smithi042.stdout:Available: Yes 2023-12-12T09:58:01.459 INFO:teuthology.orchestra.run.smithi042.stdout:Paused: No 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.458+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8e440a1570 msgr2=0x7f8e440a3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.458+0000 7f8e618d5700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8e440a1570 0x7f8e440a3a30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8e50009990 tx=0x7f8e50005ad0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.458+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c100520 msgr2=0x7f8e5c199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.458+0000 7f8e618d5700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c100520 0x7f8e5c199720 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00d9b0 tx=0x7f8e4c00dcc0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.459+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 shutdown_connections 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.459+0000 7f8e618d5700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f8e440a1570 0x7f8e440a3a30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.459+0000 7f8e618d5700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c1020c0 0x7f8e5c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.459+0000 7f8e618d5700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8e5c100520 0x7f8e5c199720 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.459+0000 7f8e618d5700 1 --2- 172.21.15.42:0/2511330465 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e5c101720 0x7f8e5c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.459+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 >> 172.21.15.42:0/2511330465 conn(0x7f8e5c0fbae0 msgr2=0x7f8e5c190490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.459+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 shutdown_connections 2023-12-12T09:58:01.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:01.459+0000 7f8e618d5700 1 -- 172.21.15.42:0/2511330465 wait complete. 2023-12-12T09:58:01.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:01 smithi111 ceph-mon[102565]: pgmap v305: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:01.818 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:01 smithi042 ceph-mon[98894]: pgmap v305: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:01.819 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- bash -c 'ceph orch ps' 2023-12-12T09:58:01.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:01 smithi173 ceph-mon[103308]: pgmap v305: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:02.424 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:58:02.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:02 smithi111 ceph-mon[102565]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:02.820 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:02 smithi042 ceph-mon[98894]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:02.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:02 smithi173 ceph-mon[103308]: from='client.15294 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.586+0000 7f662944f700 1 -- 172.21.15.42:0/534395763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66241004b0 msgr2=0x7f66241008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.586+0000 7f662944f700 1 --2- 172.21.15.42:0/534395763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66241004b0 0x7f66241008d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f66140099c0 tx=0x7f6614009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.586+0000 7f662944f700 1 -- 172.21.15.42:0/534395763 shutdown_connections 2023-12-12T09:58:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.586+0000 7f662944f700 1 --2- 172.21.15.42:0/534395763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6624102050 0x7f6624108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.586+0000 7f662944f700 1 --2- 172.21.15.42:0/534395763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66241004b0 0x7f66241008d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.586+0000 7f662944f700 1 --2- 172.21.15.42:0/534395763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66241016b0 0x7f6624101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.586+0000 7f662944f700 1 -- 172.21.15.42:0/534395763 >> 172.21.15.42:0/534395763 conn(0x7f66240fba50 msgr2=0x7f66240fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.586+0000 7f662944f700 1 -- 172.21.15.42:0/534395763 shutdown_connections 2023-12-12T09:58:03.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.587+0000 7f662944f700 1 -- 172.21.15.42:0/534395763 wait complete. 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.587+0000 7f662944f700 1 Processor -- start 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.587+0000 7f662944f700 1 -- start start 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f662944f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66241004b0 0x7f6624069a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f662944f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66241016b0 0x7f662419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f662944f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6624102050 0x7f662419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f662944f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419ee60 con 0x7f6624102050 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f662944f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419efd0 con 0x7f66241016b0 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f662944f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662419f190 con 0x7f66241004b0 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66237fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6624102050 0x7f662419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66237fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6624102050 0x7f662419e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34886/0 (socket says 172.21.15.42:34886) 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66237fe700 1 -- 172.21.15.42:0/1026150806 learned_addr learned my addr 172.21.15.42:0/1026150806 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f6622ffd700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66241004b0 0x7f6624069a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:03.592 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66227fc700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66241016b0 0x7f662419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66237fe700 1 -- 172.21.15.42:0/1026150806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66241004b0 msgr2=0x7f6624069a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66237fe700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66241004b0 0x7f6624069a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66237fe700 1 -- 172.21.15.42:0/1026150806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66241016b0 msgr2=0x7f662419c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66237fe700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66241016b0 0x7f662419c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66237fe700 1 -- 172.21.15.42:0/1026150806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140095f0 con 0x7f6624102050 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66227fc700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66241016b0 0x7f662419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f66237fe700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6624102050 0x7f662419e6c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f661c007d70 tx=0x7f661c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f660bfff700 1 -- 172.21.15.42:0/1026150806 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c00ad40 con 0x7f6624102050 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662419f470 con 0x7f6624102050 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f660bfff700 1 -- 172.21.15.42:0/1026150806 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f661c0128e0 con 0x7f6624102050 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.588+0000 7f660bfff700 1 -- 172.21.15.42:0/1026150806 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c0116b0 con 0x7f6624102050 2023-12-12T09:58:03.593 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.589+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662419f970 con 0x7f6624102050 2023-12-12T09:58:03.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.590+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66241078c0 con 0x7f6624102050 2023-12-12T09:58:03.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.590+0000 7f660bfff700 1 -- 172.21.15.42:0/1026150806 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f661c012420 con 0x7f6624102050 2023-12-12T09:58:03.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.591+0000 7f660bfff700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f660c0a1590 0x7f660c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:03.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.591+0000 7f6622ffd700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f660c0a1590 0x7f660c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:03.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.591+0000 7f660bfff700 1 -- 172.21.15.42:0/1026150806 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f661c0cd2c0 con 0x7f6624102050 2023-12-12T09:58:03.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.592+0000 7f6622ffd700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f660c0a1590 0x7f660c0a3a50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6614003820 tx=0x7f6614005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:03.597 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.595+0000 7f660bfff700 1 -- 172.21.15.42:0/1026150806 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f661c0965c0 con 0x7f6624102050 2023-12-12T09:58:03.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:03 smithi042 ceph-mon[98894]: pgmap v306: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:03.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:03 smithi111 ceph-mon[102565]: pgmap v306: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:03.806 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.804+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6624061a10 con 0x7f660c0a1590 2023-12-12T09:58:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:03 smithi173 ceph-mon[103308]: pgmap v306: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:03.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.825+0000 7f660bfff700 1 -- 172.21.15.42:0/1026150806 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4123 (secure 0 0 0) 0x7f6624061a10 con 0x7f660c0a1590 2023-12-12T09:58:03.827 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.a smithi042 running (94s) 17s ago 93s 17.4M - 19.0.0-96-gf1fbf3c5 8cf689f388db 7e3ddada8fac 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.b smithi111 running (91s) 21s ago 91s 31.1M - 19.0.0-96-gf1fbf3c5 8cf689f388db 24b6f60d4978 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.c smithi173 running (103s) 21s ago 103s 20.6M - 19.0.0-96-gf1fbf3c5 8cf689f388db 0b85d811b689 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.d smithi042 running (109s) 17s ago 109s 16.6M - 19.0.0-96-gf1fbf3c5 8cf689f388db ca9a3af31db2 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.e smithi111 running (107s) 21s ago 107s 18.7M - 19.0.0-96-gf1fbf3c5 8cf689f388db 98f0622a5996 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.f smithi173 running (105s) 21s ago 105s 18.7M - 19.0.0-96-gf1fbf3c5 8cf689f388db 1b95c1c40dde 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.g smithi042 running (96s) 17s ago 96s 20.2M - 19.0.0-96-gf1fbf3c5 8cf689f388db b19dca8b6377 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.h smithi111 running (101s) 21s ago 100s 17.9M - 19.0.0-96-gf1fbf3c5 8cf689f388db 9484396f74d3 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.i smithi173 running (99s) 21s ago 98s 21.9M - 19.0.0-96-gf1fbf3c5 8cf689f388db 7f9ddbf27eec 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.j smithi042 running (112s) 17s ago 111s 15.2M - 19.0.0-96-gf1fbf3c5 8cf689f388db 2ecb72fc0275 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.k smithi111 running (114s) 21s ago 114s 20.0M - 19.0.0-96-gf1fbf3c5 8cf689f388db 6622f742790b 2023-12-12T09:58:03.828 INFO:teuthology.orchestra.run.smithi042.stdout:mds.l smithi173 running (116s) 21s ago 116s 19.0M - 19.0.0-96-gf1fbf3c5 8cf689f388db f79e683d4101 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.x smithi042 *:9283,8765 running (8m) 17s ago 8m 526M - 19.0.0-96-gf1fbf3c5 8cf689f388db 13967c32f53e 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.y smithi111 *:8443,8765 running (6m) 21s ago 6m 451M - 19.0.0-96-gf1fbf3c5 8cf689f388db 675e2f66650d 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.z smithi173 *:8443,8765 running (6m) 21s ago 6m 450M - 19.0.0-96-gf1fbf3c5 8cf689f388db f03296794077 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a smithi042 running (8m) 17s ago 8m 94.7M 2048M 19.0.0-96-gf1fbf3c5 8cf689f388db 9b13c8f8a6fa 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b smithi111 running (6m) 21s ago 6m 63.0M 2048M 19.0.0-96-gf1fbf3c5 8cf689f388db dc6d9382b8ef 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c smithi173 running (6m) 21s ago 6m 65.5M 2048M 19.0.0-96-gf1fbf3c5 8cf689f388db 347f73327bef 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (6m) 17s ago 6m 74.3M 4293M 19.0.0-96-gf1fbf3c5 8cf689f388db 0bad15fe9ec4 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi111 running (5m) 21s ago 5m 77.7M 4292M 19.0.0-96-gf1fbf3c5 8cf689f388db cfc33704d54c 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi173 running (5m) 21s ago 5m 78.0M 4292M 19.0.0-96-gf1fbf3c5 8cf689f388db 08e632783743 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi042 running (5m) 17s ago 5m 66.7M 4293M 19.0.0-96-gf1fbf3c5 8cf689f388db ebdf9f5b8897 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi111 running (4m) 21s ago 4m 64.4M 4292M 19.0.0-96-gf1fbf3c5 8cf689f388db 5d15a3417935 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi173 running (4m) 21s ago 4m 72.6M 4292M 19.0.0-96-gf1fbf3c5 8cf689f388db dfbdee57c546 2023-12-12T09:58:03.829 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 smithi042 running (4m) 17s ago 4m 61.6M 4293M 19.0.0-96-gf1fbf3c5 8cf689f388db 003ed3517cfd 2023-12-12T09:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 smithi111 running (3m) 21s ago 3m 68.8M 4292M 19.0.0-96-gf1fbf3c5 8cf689f388db a473888b6cd0 2023-12-12T09:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 smithi173 running (3m) 21s ago 3m 67.1M 4292M 19.0.0-96-gf1fbf3c5 8cf689f388db 70ecc951dd6c 2023-12-12T09:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 smithi042 running (2m) 17s ago 2m 61.9M 4293M 19.0.0-96-gf1fbf3c5 8cf689f388db fa0432e68dc7 2023-12-12T09:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 smithi111 running (2m) 21s ago 2m 63.1M 4292M 19.0.0-96-gf1fbf3c5 8cf689f388db 806357dd74ad 2023-12-12T09:58:03.830 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 smithi173 running (2m) 21s ago 2m 56.7M 4292M 19.0.0-96-gf1fbf3c5 8cf689f388db 9cabe907d6f6 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f660c0a1590 msgr2=0x7f660c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f660c0a1590 0x7f660c0a3a50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6614003820 tx=0x7f6614005040 comp rx=0 tx=0).stop 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6624102050 msgr2=0x7f662419e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6624102050 0x7f662419e6c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f661c007d70 tx=0x7f661c00a340 comp rx=0 tx=0).stop 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 shutdown_connections 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f660c0a1590 0x7f660c0a3a50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6624102050 0x7f662419e6c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66241004b0 0x7f6624069a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 --2- 172.21.15.42:0/1026150806 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66241016b0 0x7f662419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:03.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.827+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 >> 172.21.15.42:0/1026150806 conn(0x7f66240fba50 msgr2=0x7f6624104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:03.832 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.828+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 shutdown_connections 2023-12-12T09:58:03.832 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:03.828+0000 7f662944f700 1 -- 172.21.15.42:0/1026150806 wait complete. 2023-12-12T09:58:04.401 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- bash -c 'ceph orch ls' 2023-12-12T09:58:05.055 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:58:05.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:05 smithi042 ceph-mon[98894]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:05.540 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:05 smithi042 ceph-mon[98894]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:05 smithi111 ceph-mon[102565]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:05.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:05 smithi111 ceph-mon[102565]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:05 smithi173 ceph-mon[103308]: from='client.15300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:05.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:05 smithi173 ceph-mon[103308]: pgmap v307: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:06.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.236+0000 7f9d11aa9700 1 -- 172.21.15.42:0/2173474338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1020a0 msgr2=0x7f9d0c104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:06.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.236+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/2173474338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1020a0 0x7f9d0c104490 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009990 tx=0x7f9cfc009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.236+0000 7f9d11aa9700 1 -- 172.21.15.42:0/2173474338 shutdown_connections 2023-12-12T09:58:06.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.236+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/2173474338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0c107300 0x7f9d0c109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.236+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/2173474338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1020a0 0x7f9d0c104490 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.236+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/2173474338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d0c1049d0 0x7f9d0c106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.236+0000 7f9d11aa9700 1 -- 172.21.15.42:0/2173474338 >> 172.21.15.42:0/2173474338 conn(0x7f9d0c0fbad0 msgr2=0x7f9d0c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:06.241 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.236+0000 7f9d11aa9700 1 -- 172.21.15.42:0/2173474338 shutdown_connections 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.236+0000 7f9d11aa9700 1 -- 172.21.15.42:0/2173474338 wait complete. 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.237+0000 7f9d11aa9700 1 Processor -- start 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.237+0000 7f9d11aa9700 1 -- start start 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.237+0000 7f9d11aa9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d0c1020a0 0x7f9d0c1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.237+0000 7f9d11aa9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1049d0 0x7f9d0c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.237+0000 7f9d11aa9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0c107300 0x7f9d0c19ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.237+0000 7f9d11aa9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c19eff0 con 0x7f9d0c1049d0 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.237+0000 7f9d11aa9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c19f130 con 0x7f9d0c1020a0 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.237+0000 7f9d11aa9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0c19f2c0 con 0x7f9d0c107300 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.237+0000 7f9d0affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1049d0 0x7f9d0c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1049d0 0x7f9d0c1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34898/0 (socket says 172.21.15.42:34898) 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0affd700 1 -- 172.21.15.42:0/676094339 learned_addr learned my addr 172.21.15.42:0/676094339 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:06.242 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0bfff700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0c107300 0x7f9d0c19ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0b7fe700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d0c1020a0 0x7f9d0c1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0affd700 1 -- 172.21.15.42:0/676094339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0c107300 msgr2=0x7f9d0c19ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0affd700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0c107300 0x7f9d0c19ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0affd700 1 -- 172.21.15.42:0/676094339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d0c1020a0 msgr2=0x7f9d0c1974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0affd700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d0c1020a0 0x7f9d0c1974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0affd700 1 -- 172.21.15.42:0/676094339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc0095f0 con 0x7f9d0c1049d0 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0b7fe700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d0c1020a0 0x7f9d0c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0bfff700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0c107300 0x7f9d0c19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d0affd700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1049d0 0x7f9d0c1979f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c9e0 tx=0x7f9d0000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d08ff9700 1 -- 172.21.15.42:0/676094339 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00007ac0 con 0x7f9d0c1049d0 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d08ff9700 1 -- 172.21.15.42:0/676094339 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d00007c20 con 0x7f9d0c1049d0 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0c19f5a0 con 0x7f9d0c1049d0 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d08ff9700 1 -- 172.21.15.42:0/676094339 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00018660 con 0x7f9d0c1049d0 2023-12-12T09:58:06.243 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.238+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0c19faa0 con 0x7f9d0c1049d0 2023-12-12T09:58:06.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.239+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0c0fd650 con 0x7f9d0c1049d0 2023-12-12T09:58:06.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.240+0000 7f9d08ff9700 1 -- 172.21.15.42:0/676094339 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9d0000f450 con 0x7f9d0c1049d0 2023-12-12T09:58:06.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.241+0000 7f9d08ff9700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9cf40a1510 0x7f9cf40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:06.244 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.241+0000 7f9d0b7fe700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9cf40a1510 0x7f9cf40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:06.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.241+0000 7f9d08ff9700 1 -- 172.21.15.42:0/676094339 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9d000cd1a0 con 0x7f9d0c1049d0 2023-12-12T09:58:06.245 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.241+0000 7f9d0b7fe700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9cf40a1510 0x7f9cf40a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0037f0 tx=0x7f9cfc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:06.247 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.245+0000 7f9d08ff9700 1 -- 172.21.15.42:0/676094339 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9d00096390 con 0x7f9d0c1049d0 2023-12-12T09:58:06.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.459+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d0c02d380 con 0x7f9cf40a1510 2023-12-12T09:58:06.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.467+0000 7f9d08ff9700 1 -- 172.21.15.42:0/676094339 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7f9d0c02d380 con 0x7f9cf40a1510 2023-12-12T09:58:06.469 INFO:teuthology.orchestra.run.smithi042.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-12-12T09:58:06.470 INFO:teuthology.orchestra.run.smithi042.stdout:mds.all 12/12 24s ago 2m smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi111=b;smithi111=e;smithi111=h;smithi111=k;smithi173=c;smithi173=f;smithi173=i;smithi173=l;count:12 2023-12-12T09:58:06.470 INFO:teuthology.orchestra.run.smithi042.stdout:mgr 3/3 24s ago 6m smithi042=x;smithi111=y;smithi173=z;count:3 2023-12-12T09:58:06.470 INFO:teuthology.orchestra.run.smithi042.stdout:mon 3/3 24s ago 6m smithi042:172.21.15.42=a;smithi111:172.21.15.111=b;smithi173:172.21.15.173=c;count:3 2023-12-12T09:58:06.470 INFO:teuthology.orchestra.run.smithi042.stdout:osd 12 24s ago - 2023-12-12T09:58:06.471 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9cf40a1510 msgr2=0x7f9cf40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:06.471 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9cf40a1510 0x7f9cf40a39d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0037f0 tx=0x7f9cfc005fb0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1049d0 msgr2=0x7f9d0c1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1049d0 0x7f9d0c1979f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c9e0 tx=0x7f9d0000cda0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 shutdown_connections 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f9cf40a1510 0x7f9cf40a39d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d0c1020a0 0x7f9d0c1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0c107300 0x7f9d0c19ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 --2- 172.21.15.42:0/676094339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d0c1049d0 0x7f9d0c1979f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.469+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 >> 172.21.15.42:0/676094339 conn(0x7f9d0c0fbad0 msgr2=0x7f9d0c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.470+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 shutdown_connections 2023-12-12T09:58:06.472 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:06.470+0000 7f9d11aa9700 1 -- 172.21.15.42:0/676094339 wait complete. 2023-12-12T09:58:06.930 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- bash -c 'ceph orch host ls' 2023-12-12T09:58:07.533 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:58:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:07 smithi111 ceph-mon[102565]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:07.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:07 smithi111 ceph-mon[102565]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:07.753 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:07 smithi042 ceph-mon[98894]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:07.754 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:07 smithi042 ceph-mon[98894]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:07.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:07 smithi173 ceph-mon[103308]: pgmap v308: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:07.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:07 smithi173 ceph-mon[103308]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:08.794 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.790+0000 7faac9aac700 1 -- 172.21.15.42:0/3224782131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4102820 msgr2=0x7faac4102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.790+0000 7faac9aac700 1 --2- 172.21.15.42:0/3224782131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4102820 0x7faac4102c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7faab40099c0 tx=0x7faab4009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.790+0000 7faac9aac700 1 -- 172.21.15.42:0/3224782131 shutdown_connections 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.790+0000 7faac9aac700 1 --2- 172.21.15.42:0/3224782131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac41031c0 0x7faac4109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.790+0000 7faac9aac700 1 --2- 172.21.15.42:0/3224782131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac4101620 0x7faac4101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.790+0000 7faac9aac700 1 --2- 172.21.15.42:0/3224782131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4102820 0x7faac4102c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.790+0000 7faac9aac700 1 -- 172.21.15.42:0/3224782131 >> 172.21.15.42:0/3224782131 conn(0x7faac40fcbc0 msgr2=0x7faac40ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.790+0000 7faac9aac700 1 -- 172.21.15.42:0/3224782131 shutdown_connections 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.790+0000 7faac9aac700 1 -- 172.21.15.42:0/3224782131 wait complete. 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.791+0000 7faac9aac700 1 Processor -- start 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.791+0000 7faac9aac700 1 -- start start 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.791+0000 7faac9aac700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4101620 0x7faac4197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:08.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.791+0000 7faac9aac700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac4102820 0x7faac4197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.791+0000 7faac9aac700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac41031c0 0x7faac419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.791+0000 7faac9aac700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac419f070 con 0x7faac4101620 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.791+0000 7faac9aac700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac419f1b0 con 0x7faac41031c0 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.791+0000 7faac9aac700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac419f340 con 0x7faac4102820 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.791+0000 7faac37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4101620 0x7faac4197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4101620 0x7faac4197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34910/0 (socket says 172.21.15.42:34910) 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac37fe700 1 -- 172.21.15.42:0/1699480570 learned_addr learned my addr 172.21.15.42:0/1699480570 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac2ffd700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac4102820 0x7faac4197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac3fff700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac41031c0 0x7faac419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac37fe700 1 -- 172.21.15.42:0/1699480570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac4102820 msgr2=0x7faac4197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac37fe700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac4102820 0x7faac4197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac37fe700 1 -- 172.21.15.42:0/1699480570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac41031c0 msgr2=0x7faac419eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac37fe700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac41031c0 0x7faac419eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac37fe700 1 -- 172.21.15.42:0/1699480570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab40095f0 con 0x7faac4101620 2023-12-12T09:58:08.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac37fe700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4101620 0x7faac4197530 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faab800d9b0 tx=0x7faab800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:08.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac0ff9700 1 -- 172.21.15.42:0/1699480570 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8004020 con 0x7faac4101620 2023-12-12T09:58:08.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac419f620 con 0x7faac4101620 2023-12-12T09:58:08.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac0ff9700 1 -- 172.21.15.42:0/1699480570 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faab8009730 con 0x7faac4101620 2023-12-12T09:58:08.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac0ff9700 1 -- 172.21.15.42:0/1699480570 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8003930 con 0x7faac4101620 2023-12-12T09:58:08.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.792+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac419fb70 con 0x7faac4101620 2023-12-12T09:58:08.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.793+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac4108aa0 con 0x7faac4101620 2023-12-12T09:58:08.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.795+0000 7faac0ff9700 1 -- 172.21.15.42:0/1699480570 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7faab80098a0 con 0x7faac4101620 2023-12-12T09:58:08.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.796+0000 7faac0ff9700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faaac0a1510 0x7faaac0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:08.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.796+0000 7faac2ffd700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faaac0a1510 0x7faaac0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:08.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.796+0000 7faac0ff9700 1 -- 172.21.15.42:0/1699480570 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faab80cd170 con 0x7faac4101620 2023-12-12T09:58:08.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.796+0000 7faac2ffd700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faaac0a1510 0x7faaac0a39d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faab4009990 tx=0x7faab4005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:08.801 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:08.799+0000 7faac0ff9700 1 -- 172.21.15.42:0/1699480570 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7faab8096300 con 0x7faac4101620 2023-12-12T09:58:09.010 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.008+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7faac4061a10 con 0x7faaac0a1510 2023-12-12T09:58:09.012 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.010+0000 7faac0ff9700 1 -- 172.21.15.42:0/1699480570 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7faac4061a10 con 0x7faaac0a1510 2023-12-12T09:58:09.012 INFO:teuthology.orchestra.run.smithi042.stdout:HOST ADDR LABELS STATUS 2023-12-12T09:58:09.012 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 172.21.15.42 2023-12-12T09:58:09.012 INFO:teuthology.orchestra.run.smithi042.stdout:smithi111 172.21.15.111 2023-12-12T09:58:09.012 INFO:teuthology.orchestra.run.smithi042.stdout:smithi173 172.21.15.173 2023-12-12T09:58:09.012 INFO:teuthology.orchestra.run.smithi042.stdout:3 hosts in cluster 2023-12-12T09:58:09.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faaac0a1510 msgr2=0x7faaac0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faaac0a1510 0x7faaac0a39d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7faab4009990 tx=0x7faab4005ad0 comp rx=0 tx=0).stop 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4101620 msgr2=0x7faac4197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4101620 0x7faac4197530 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faab800d9b0 tx=0x7faab800dcc0 comp rx=0 tx=0).stop 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 shutdown_connections 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7faaac0a1510 0x7faaac0a39d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faac41031c0 0x7faac419eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faac4101620 0x7faac4197530 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 --2- 172.21.15.42:0/1699480570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faac4102820 0x7faac4197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 >> 172.21.15.42:0/1699480570 conn(0x7faac40fcbc0 msgr2=0x7faac4105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 shutdown_connections 2023-12-12T09:58:09.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:09.012+0000 7faac9aac700 1 -- 172.21.15.42:0/1699480570 wait complete. 2023-12-12T09:58:09.504 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- bash -c 'ceph orch device ls' 2023-12-12T09:58:09.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:09 smithi111 ceph-mon[102565]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:58:09.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:09 smithi173 ceph-mon[103308]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:58:09.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:09 smithi042 ceph-mon[98894]: pgmap v309: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:58:10.108 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:58:10.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:10 smithi042 ceph-mon[98894]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:10.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:10 smithi111 ceph-mon[102565]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:10.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:10 smithi173 ceph-mon[103308]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.453+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/2408343204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f41020b0 msgr2=0x7fa0f4108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.453+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/2408343204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f41020b0 0x7fa0f4108960 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00b0d0 tx=0x7fa0ec00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.454+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/2408343204 shutdown_connections 2023-12-12T09:58:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.454+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/2408343204 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f41020b0 0x7fa0f4108960 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.454+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/2408343204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0f4100510 0x7fa0f4100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.459 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.454+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/2408343204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f4101710 0x7fa0f4101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.454+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/2408343204 >> 172.21.15.42:0/2408343204 conn(0x7fa0f40fbad0 msgr2=0x7fa0f40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.454+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/2408343204 shutdown_connections 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.454+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/2408343204 wait complete. 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.454+0000 7fa0fc3f3700 1 Processor -- start 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.454+0000 7fa0fc3f3700 1 -- start start 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fc3f3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0f4100510 0x7fa0f4069a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fc3f3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f4101710 0x7fa0f419c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fc3f3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f41020b0 0x7fa0f419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fc3f3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f419f0f0 con 0x7fa0f4101710 2023-12-12T09:58:11.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fc3f3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f419f260 con 0x7fa0f41020b0 2023-12-12T09:58:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fc3f3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f419f420 con 0x7fa0f4100510 2023-12-12T09:58:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0f998e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f4101710 0x7fa0f419c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0f998e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f4101710 0x7fa0f419c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34926/0 (socket says 172.21.15.42:34926) 2023-12-12T09:58:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0f998e700 1 -- 172.21.15.42:0/602495441 learned_addr learned my addr 172.21.15.42:0/602495441 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fa18f700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0f4100510 0x7fa0f4069a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fa990700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f41020b0 0x7fa0f419e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0f998e700 1 -- 172.21.15.42:0/602495441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0f4100510 msgr2=0x7fa0f4069a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:11.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0f998e700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0f4100510 0x7fa0f4069a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0f998e700 1 -- 172.21.15.42:0/602495441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f41020b0 msgr2=0x7fa0f419e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0f998e700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f41020b0 0x7fa0f419e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0f998e700 1 -- 172.21.15.42:0/602495441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec009d00 con 0x7fa0f4101710 2023-12-12T09:58:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fa990700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f41020b0 0x7fa0f419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0f998e700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f4101710 0x7fa0f419c3c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c9b0 tx=0x7fa0e400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:11.462 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0eb7fe700 1 -- 172.21.15.42:0/602495441 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4007b00 con 0x7fa0f4101710 2023-12-12T09:58:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0eb7fe700 1 -- 172.21.15.42:0/602495441 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0e4007c60 con 0x7fa0f4101710 2023-12-12T09:58:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f419f700 con 0x7fa0f4101710 2023-12-12T09:58:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.455+0000 7fa0eb7fe700 1 -- 172.21.15.42:0/602495441 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4018660 con 0x7fa0f4101710 2023-12-12T09:58:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.456+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f419fc50 con 0x7fa0f4101710 2023-12-12T09:58:11.463 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.457+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f41078d0 con 0x7fa0f4101710 2023-12-12T09:58:11.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.458+0000 7fa0eb7fe700 1 -- 172.21.15.42:0/602495441 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa0e400f450 con 0x7fa0f4101710 2023-12-12T09:58:11.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.458+0000 7fa0eb7fe700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa0e00a1590 0x7fa0e00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.458+0000 7fa0eb7fe700 1 -- 172.21.15.42:0/602495441 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa0e40cd300 con 0x7fa0f4101710 2023-12-12T09:58:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.458+0000 7fa0fa18f700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa0e00a1590 0x7fa0e00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.459+0000 7fa0fa18f700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa0e00a1590 0x7fa0e00a3a50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4101570 tx=0x7fa0f0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:11.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.462+0000 7fa0eb7fe700 1 -- 172.21.15.42:0/602495441 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa0e40966a0 con 0x7fa0f4101710 2023-12-12T09:58:11.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:11 smithi042 ceph-mon[98894]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:11.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.670+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0f402d380 con 0x7fa0e00a1590 2023-12-12T09:58:11.674 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.672+0000 7fa0eb7fe700 1 -- 172.21.15.42:0/602495441 <== mgr.14152 v2:172.21.15.42:6800/2637927159 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7fa0f402d380 con 0x7fa0e00a1590 2023-12-12T09:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-12-12T09:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130021480DGN 447G No 2m ago Has a FileSystem, LVM detected 2023-12-12T09:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout:smithi111 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400VZ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-12T09:58:11.675 INFO:teuthology.orchestra.run.smithi042.stdout:smithi173 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204019S400BGN 372G No 2m ago Has a FileSystem, LVM detected 2023-12-12T09:58:11.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa0e00a1590 msgr2=0x7fa0e00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:11.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa0e00a1590 0x7fa0e00a3a50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4101570 tx=0x7fa0f0009190 comp rx=0 tx=0).stop 2023-12-12T09:58:11.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f4101710 msgr2=0x7fa0f419c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:11.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f4101710 0x7fa0f419c3c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c9b0 tx=0x7fa0e400ccc0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 shutdown_connections 2023-12-12T09:58:11.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa0e00a1590 0x7fa0e00a3a50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0f41020b0 0x7fa0f419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0f4100510 0x7fa0f4069a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 --2- 172.21.15.42:0/602495441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0f4101710 0x7fa0f419c3c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:11.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 >> 172.21.15.42:0/602495441 conn(0x7fa0f40fbad0 msgr2=0x7fa0f4104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:11.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 shutdown_connections 2023-12-12T09:58:11.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:11.674+0000 7fa0fc3f3700 1 -- 172.21.15.42:0/602495441 wait complete. 2023-12-12T09:58:11.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:11 smithi111 ceph-mon[102565]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:11.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:11 smithi173 ceph-mon[103308]: pgmap v310: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:12.086 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-12T09:58:12.098 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T09:58:12.098 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- bash -c 'ceph fs dump' 2023-12-12T09:58:12.703 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:58:12.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:12 smithi111 ceph-mon[102565]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:12.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:12 smithi173 ceph-mon[103308]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:12.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:12 smithi042 ceph-mon[98894]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:13.738 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:13 smithi042 ceph-mon[98894]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:13.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:13 smithi111 ceph-mon[102565]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:13.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:13 smithi173 ceph-mon[103308]: pgmap v311: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.909+0000 7f2413bca700 1 -- 172.21.15.42:0/258711968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c101710 msgr2=0x7f240c101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.909+0000 7f2413bca700 1 --2- 172.21.15.42:0/258711968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c101710 0x7f240c101b70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f24000099c0 tx=0x7f2400009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.910+0000 7f2413bca700 1 -- 172.21.15.42:0/258711968 shutdown_connections 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.910+0000 7f2413bca700 1 --2- 172.21.15.42:0/258711968 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c1020b0 0x7f240c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.910+0000 7f2413bca700 1 --2- 172.21.15.42:0/258711968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f240c100510 0x7f240c100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.910+0000 7f2413bca700 1 --2- 172.21.15.42:0/258711968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c101710 0x7f240c101b70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.910+0000 7f2413bca700 1 -- 172.21.15.42:0/258711968 >> 172.21.15.42:0/258711968 conn(0x7f240c0fbad0 msgr2=0x7f240c0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.910+0000 7f2413bca700 1 -- 172.21.15.42:0/258711968 shutdown_connections 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.910+0000 7f2413bca700 1 -- 172.21.15.42:0/258711968 wait complete. 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.910+0000 7f2413bca700 1 Processor -- start 2023-12-12T09:58:13.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.910+0000 7f2413bca700 1 -- start start 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2413bca700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c100510 0x7f240c197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2413bca700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f240c101710 0x7f240c197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2413bca700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c1020b0 0x7f240c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2413bca700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19f090 con 0x7f240c100510 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2413bca700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19f1d0 con 0x7f240c1020b0 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2413bca700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c19f360 con 0x7f240c101710 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411966700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c100510 0x7f240c197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411966700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c100510 0x7f240c197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34938/0 (socket says 172.21.15.42:34938) 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411966700 1 -- 172.21.15.42:0/339999313 learned_addr learned my addr 172.21.15.42:0/339999313 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411165700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f240c101710 0x7f240c197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2412167700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c1020b0 0x7f240c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411966700 1 -- 172.21.15.42:0/339999313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f240c101710 msgr2=0x7f240c197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411966700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f240c101710 0x7f240c197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:13.915 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411966700 1 -- 172.21.15.42:0/339999313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c1020b0 msgr2=0x7f240c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411966700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c1020b0 0x7f240c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411966700 1 -- 172.21.15.42:0/339999313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24000095f0 con 0x7f240c100510 2023-12-12T09:58:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2411966700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c100510 0x7f240c197550 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f240800d9b0 tx=0x7f240800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f23feffd700 1 -- 172.21.15.42:0/339999313 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408004020 con 0x7f240c100510 2023-12-12T09:58:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f23feffd700 1 -- 172.21.15.42:0/339999313 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2408009730 con 0x7f240c100510 2023-12-12T09:58:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f23feffd700 1 -- 172.21.15.42:0/339999313 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408003a30 con 0x7f240c100510 2023-12-12T09:58:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.911+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c19f640 con 0x7f240c100510 2023-12-12T09:58:13.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.912+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c19fb40 con 0x7f240c100510 2023-12-12T09:58:13.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.913+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c04f290 con 0x7f240c100510 2023-12-12T09:58:13.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.914+0000 7f23feffd700 1 -- 172.21.15.42:0/339999313 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f24080098a0 con 0x7f240c100510 2023-12-12T09:58:13.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.914+0000 7f23feffd700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f23f80a1660 0x7f23f80a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:13.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.914+0000 7f2411165700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f23f80a1660 0x7f23f80a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:13.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.914+0000 7f23feffd700 1 -- 172.21.15.42:0/339999313 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f24080ccd70 con 0x7f240c100510 2023-12-12T09:58:13.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.915+0000 7f2411165700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f23f80a1660 0x7f23f80a3b20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2400009990 tx=0x7f2400005ad0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:13.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:13.919+0000 7f23feffd700 1 -- 172.21.15.42:0/339999313 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2408095f60 con 0x7f240c100510 2023-12-12T09:58:14.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.162+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f240c10bd20 con 0x7f240c100510 2023-12-12T09:58:14.164 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.162+0000 7f23feffd700 1 -- 172.21.15.42:0/339999313 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2981 (secure 0 0 0) 0x7f2408017bc0 con 0x7f240c100510 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:e27 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 27 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-12-12T09:58:14.165 INFO:teuthology.orchestra.run.smithi042.stdout:created 2023-12-12T09:56:37.902338+0000 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2023-12-12T09:57:07.293702+0000 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 300 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 5 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1,2,3,4 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24524,1=24508,2=24487,3=24482,4=24475} 2023-12-12T09:58:14.166 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3,4] 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 0 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b{0:24524} state up:active seq 3 addr [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e{0:24452} state up:standby-replay seq 1 addr [v2:172.21.15.111:6834/4127941301,v1:172.21.15.111:6835/4127941301] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.i{1:24508} state up:active seq 11 addr [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.l{1:24430} state up:standby-replay seq 1 addr [v2:172.21.15.173:6832/2859393185,v1:172.21.15.173:6833/2859393185] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c{2:24487} state up:active seq 12 addr [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.k{2:24425} state up:standby-replay seq 1 addr [v2:172.21.15.111:6832/1539682179,v1:172.21.15.111:6833/1539682179] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.h{3:24482} state up:active seq 12 addr [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.167 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a{3:14628} state up:standby-replay seq 1 addr [v2:172.21.15.42:6840/4221329195,v1:172.21.15.42:6841/4221329195] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.168 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.f{4:24475} state up:active seq 13 addr [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.168 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.g{4:14616} state up:standby-replay seq 1 addr [v2:172.21.15.42:6838/229824980,v1:172.21.15.42:6839/229824980] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.168 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:58:14.168 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:58:14.168 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2023-12-12T09:58:14.168 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:58:14.168 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.42:6834/4073397880,v1:172.21.15.42:6835/4073397880] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.169 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d{-1:14559} state up:standby seq 1 addr [v2:172.21.15.42:6836/3743179195,v1:172.21.15.42:6837/3743179195] compat {c=[1],r=[1],i=[fff]}] 2023-12-12T09:58:14.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f23f80a1660 msgr2=0x7f23f80a3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:14.169 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f23f80a1660 0x7f23f80a3b20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2400009990 tx=0x7f2400005ad0 comp rx=0 tx=0).stop 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c100510 msgr2=0x7f240c197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c100510 0x7f240c197550 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f240800d9b0 tx=0x7f240800dcc0 comp rx=0 tx=0).stop 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 shutdown_connections 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f23f80a1660 0x7f23f80a3b20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f240c1020b0 0x7f240c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f240c100510 0x7f240c197550 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 --2- 172.21.15.42:0/339999313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f240c101710 0x7f240c197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 >> 172.21.15.42:0/339999313 conn(0x7f240c0fbad0 msgr2=0x7f240c0fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 shutdown_connections 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:14.164+0000 7f2413bca700 1 -- 172.21.15.42:0/339999313 wait complete. 2023-12-12T09:58:14.170 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2023-12-12T09:58:14.578 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- bash -c 'ceph osd dump' 2023-12-12T09:58:14.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:14 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/339999313' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-12T09:58:14.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:14 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/339999313' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-12T09:58:14.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:14 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/339999313' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-12T09:58:15.180 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:58:15.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:15 smithi042 ceph-mon[98894]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:15.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:15 smithi111 ceph-mon[102565]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:15.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:15 smithi173 ceph-mon[103308]: pgmap v312: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.377+0000 7f55044de700 1 -- 172.21.15.42:0/1204639337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc101710 msgr2=0x7f54fc101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.377+0000 7f55044de700 1 --2- 172.21.15.42:0/1204639337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc101710 0x7f54fc101b70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f54f80099c0 tx=0x7f54f8009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.377+0000 7f55044de700 1 -- 172.21.15.42:0/1204639337 shutdown_connections 2023-12-12T09:58:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.377+0000 7f55044de700 1 --2- 172.21.15.42:0/1204639337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54fc100510 0x7f54fc100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.377+0000 7f55044de700 1 --2- 172.21.15.42:0/1204639337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc1020b0 0x7f54fc108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.383 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.377+0000 7f55044de700 1 --2- 172.21.15.42:0/1204639337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc101710 0x7f54fc101b70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.377+0000 7f55044de700 1 -- 172.21.15.42:0/1204639337 >> 172.21.15.42:0/1204639337 conn(0x7f54fc0fbad0 msgr2=0x7f54fc0fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.377+0000 7f55044de700 1 -- 172.21.15.42:0/1204639337 shutdown_connections 2023-12-12T09:58:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.377+0000 7f55044de700 1 -- 172.21.15.42:0/1204639337 wait complete. 2023-12-12T09:58:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f55044de700 1 Processor -- start 2023-12-12T09:58:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f55044de700 1 -- start start 2023-12-12T09:58:16.384 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f55044de700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc100510 0x7f54fc1974e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:16.385 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f55044de700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc101710 0x7f54fc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:16.385 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f55044de700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54fc1020b0 0x7f54fc19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:16.385 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f55044de700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19f020 con 0x7f54fc100510 2023-12-12T09:58:16.385 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f55044de700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19f160 con 0x7f54fc1020b0 2023-12-12T09:58:16.385 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f55044de700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19f2f0 con 0x7f54fc101710 2023-12-12T09:58:16.385 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f550227a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc100510 0x7f54fc1974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:16.386 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f550227a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc100510 0x7f54fc1974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34950/0 (socket says 172.21.15.42:34950) 2023-12-12T09:58:16.386 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.378+0000 7f550227a700 1 -- 172.21.15.42:0/205901271 learned_addr learned my addr 172.21.15.42:0/205901271 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:16.386 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f5501a79700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc101710 0x7f54fc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:16.386 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f550227a700 1 -- 172.21.15.42:0/205901271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc101710 msgr2=0x7f54fc197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:16.386 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f550227a700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc101710 0x7f54fc197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f550227a700 1 -- 172.21.15.42:0/205901271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54fc1020b0 msgr2=0x7f54fc19ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f5502a7b700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54fc1020b0 0x7f54fc19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f550227a700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54fc1020b0 0x7f54fc19ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f550227a700 1 -- 172.21.15.42:0/205901271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80095f0 con 0x7f54fc100510 2023-12-12T09:58:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f5502a7b700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54fc1020b0 0x7f54fc19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:16.387 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f550227a700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc100510 0x7f54fc1974e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00d9b0 tx=0x7f54ec00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f54f37fe700 1 -- 172.21.15.42:0/205901271 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00be80 con 0x7f54fc100510 2023-12-12T09:58:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f54f37fe700 1 -- 172.21.15.42:0/205901271 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54ec0044e0 con 0x7f54fc100510 2023-12-12T09:58:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc19f5d0 con 0x7f54fc100510 2023-12-12T09:58:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f54f37fe700 1 -- 172.21.15.42:0/205901271 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec003b40 con 0x7f54fc100510 2023-12-12T09:58:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.379+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc19fb20 con 0x7f54fc100510 2023-12-12T09:58:16.388 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.380+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc04f290 con 0x7f54fc100510 2023-12-12T09:58:16.390 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.381+0000 7f54f37fe700 1 -- 172.21.15.42:0/205901271 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f54ec009940 con 0x7f54fc100510 2023-12-12T09:58:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.382+0000 7f54f37fe700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f54e80a1590 0x7f54e80a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.382+0000 7f54f37fe700 1 -- 172.21.15.42:0/205901271 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f54ec0cd220 con 0x7f54fc100510 2023-12-12T09:58:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.382+0000 7f5501a79700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f54e80a1590 0x7f54e80a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.382+0000 7f5501a79700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f54e80a1590 0x7f54e80a3a50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54f8003820 tx=0x7f54f8000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:16.391 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.385+0000 7f54f37fe700 1 -- 172.21.15.42:0/205901271 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f54ec096490 con 0x7f54fc100510 2023-12-12T09:58:16.589 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.586+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f54fc10bf10 con 0x7f54fc100510 2023-12-12T09:58:16.589 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.587+0000 7f54f37fe700 1 -- 172.21.15.42:0/205901271 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v89) v1 ==== 56+0+5598 (secure 0 0 0) 0x7f54ec099ab0 con 0x7f54fc100510 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 89 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:fsid 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:created 2023-12-12T09:49:23.808107+0000 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2023-12-12T09:57:29.362715+0000 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 27 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client luminous 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release reef 2023-12-12T09:58:16.590 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 83 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 87 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 12 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6802/4116302283,v1:172.21.15.42:6803/4116302283] [v2:172.21.15.42:6804/4116302283,v1:172.21.15.42:6805/4116302283] exists,up 33717319-c6c1-4217-91a8-02927794dbce 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6800/804818401,v1:172.21.15.111:6801/804818401] [v2:172.21.15.111:6802/804818401,v1:172.21.15.111:6803/804818401] exists,up 067bc3ee-1118-46a9-beb0-cd9e9a237c25 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6800/3785124415,v1:172.21.15.173:6801/3785124415] [v2:172.21.15.173:6802/3785124415,v1:172.21.15.173:6803/3785124415] exists,up 1d209df1-c616-4f0e-8a58-5b0c1839967c 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6810/1515738461,v1:172.21.15.42:6811/1515738461] [v2:172.21.15.42:6812/1515738461,v1:172.21.15.42:6813/1515738461] exists,up 2c3fe771-613f-4d10-956b-78db77d66068 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 32 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6808/724947563,v1:172.21.15.111:6809/724947563] [v2:172.21.15.111:6810/724947563,v1:172.21.15.111:6811/724947563] exists,up 664d9eb1-f016-4943-a329-e18c4a7d9377 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 37 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6808/3342915848,v1:172.21.15.173:6809/3342915848] [v2:172.21.15.173:6810/3342915848,v1:172.21.15.173:6811/3342915848] exists,up 5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 45 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6818/288642128,v1:172.21.15.42:6819/288642128] [v2:172.21.15.42:6820/288642128,v1:172.21.15.42:6821/288642128] exists,up f1cf8ac5-f733-425d-9148-02d06c5871e5 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 50 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6816/1072744652,v1:172.21.15.111:6817/1072744652] [v2:172.21.15.111:6818/1072744652,v1:172.21.15.111:6819/1072744652] exists,up a8c52f12-79ac-43b9-81af-d02bb17b812f 2023-12-12T09:58:16.591 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 up in weight 1 up_from 57 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6816/1898117914,v1:172.21.15.173:6817/1898117914] [v2:172.21.15.173:6818/1898117914,v1:172.21.15.173:6819/1898117914] exists,up 7e1e8185-cda2-4a3c-9ff2-fc2480152394 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 up in weight 1 up_from 65 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6826/1755856143,v1:172.21.15.42:6827/1755856143] [v2:172.21.15.42:6828/1755856143,v1:172.21.15.42:6829/1755856143] exists,up 1c936a54-8440-4cbd-ba0b-79d236e99180 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 up in weight 1 up_from 70 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.111:6824/280900835,v1:172.21.15.111:6825/280900835] [v2:172.21.15.111:6826/280900835,v1:172.21.15.111:6827/280900835] exists,up 963a1538-8ad6-4832-9f32-cc7bb0642555 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 up in weight 1 up_from 78 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.173:6824/1783755959,v1:172.21.15.173:6825/1783755959] [v2:172.21.15.173:6826/1783755959,v1:172.21.15.173:6827/1783755959] exists,up 840f024c-8e3a-4949-8e31-27f28d498d95 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.b [2,1] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.f [8,11] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.10 [8,11] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.1b [2,4] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.11 [4,8] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.1d [4,6] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 4.2 [9,1] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 4.a [5,10] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 4.17 [9,8] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 4.18 [9,8] 2023-12-12T09:58:16.592 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/467026676 expires 2023-12-13T09:50:27.727252+0000 2023-12-12T09:58:16.593 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/2920147377 expires 2023-12-13T09:50:27.727252+0000 2023-12-12T09:58:16.593 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/1525180467 expires 2023-12-13T09:50:27.727252+0000 2023-12-12T09:58:16.593 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/1433305616 expires 2023-12-13T09:49:58.206187+0000 2023-12-12T09:58:16.593 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/1433305616 expires 2023-12-13T09:49:58.206187+0000 2023-12-12T09:58:16.593 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/3856214912 expires 2023-12-13T09:49:58.206187+0000 2023-12-12T09:58:16.593 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/1525180467 expires 2023-12-13T09:50:27.727252+0000 2023-12-12T09:58:16.593 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1613862950 expires 2023-12-13T09:50:27.727252+0000 2023-12-12T09:58:16.593 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/4273364865 expires 2023-12-13T09:49:58.206187+0000 2023-12-12T09:58:16.593 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/2212533654 expires 2023-12-13T09:49:58.206187+0000 2023-12-12T09:58:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f54e80a1590 msgr2=0x7f54e80a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f54e80a1590 0x7f54e80a3a50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54f8003820 tx=0x7f54f8000bc0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc100510 msgr2=0x7f54fc1974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc100510 0x7f54fc1974e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00d9b0 tx=0x7f54ec00dd70 comp rx=0 tx=0).stop 2023-12-12T09:58:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 shutdown_connections 2023-12-12T09:58:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f54e80a1590 0x7f54e80a3a50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54fc1020b0 0x7f54fc19ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54fc100510 0x7f54fc1974e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.594 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 --2- 172.21.15.42:0/205901271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f54fc101710 0x7f54fc197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:16.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 >> 172.21.15.42:0/205901271 conn(0x7f54fc0fbad0 msgr2=0x7f54fc068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:16.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 shutdown_connections 2023-12-12T09:58:16.595 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:16.590+0000 7f55044de700 1 -- 172.21.15.42:0/205901271 wait complete. 2023-12-12T09:58:17.053 INFO:teuthology.run_tasks:Running task fs.ready... 2023-12-12T09:58:17.071 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:58:17.204 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.199+0000 7fa0b813c700 1 -- 172.21.15.42:0/893962100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0100380 msgr2=0x7fa0b01007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.199+0000 7fa0b813c700 1 --2- 172.21.15.42:0/893962100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0100380 0x7fa0b01007a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00b0d0 tx=0x7fa0ac00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 -- 172.21.15.42:0/893962100 shutdown_connections 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 --2- 172.21.15.42:0/893962100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0b0101f10 0x7fa0b01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 --2- 172.21.15.42:0/893962100 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0100380 0x7fa0b01007a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 --2- 172.21.15.42:0/893962100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0b0101570 0x7fa0b01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 -- 172.21.15.42:0/893962100 >> 172.21.15.42:0/893962100 conn(0x7fa0b00fb8f0 msgr2=0x7fa0b00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 -- 172.21.15.42:0/893962100 shutdown_connections 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 -- 172.21.15.42:0/893962100 wait complete. 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 Processor -- start 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 -- start start 2023-12-12T09:58:17.205 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0b0100380 0x7fa0b01042c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0101570 0x7fa0b0102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0b0101f10 0x7fa0b0102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b0103630 con 0x7fa0b0101570 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b01037a0 con 0x7fa0b0100380 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.200+0000 7fa0b813c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0b01a1620 con 0x7fa0b0101f10 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b56d7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0101570 0x7fa0b0102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b56d7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0101570 0x7fa0b0102950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34966/0 (socket says 172.21.15.42:34966) 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b56d7700 1 -- 172.21.15.42:0/95767669 learned_addr learned my addr 172.21.15.42:0/95767669 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b66d9700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0b0101f10 0x7fa0b0102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.206 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b5ed8700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0b0100380 0x7fa0b01042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b56d7700 1 -- 172.21.15.42:0/95767669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0b0101f10 msgr2=0x7fa0b0102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b56d7700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0b0101f10 0x7fa0b0102e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b56d7700 1 -- 172.21.15.42:0/95767669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0b0100380 msgr2=0x7fa0b01042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b56d7700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0b0100380 0x7fa0b01042c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b56d7700 1 -- 172.21.15.42:0/95767669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac009d00 con 0x7fa0b0101570 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b66d9700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0b0101f10 0x7fa0b0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b56d7700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0101570 0x7fa0b0102950 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d950 tx=0x7fa0a000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0a6ffd700 1 -- 172.21.15.42:0/95767669 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0009ae0 con 0x7fa0b0101570 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0a6ffd700 1 -- 172.21.15.42:0/95767669 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0a0009c40 con 0x7fa0b0101570 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b01a18b0 con 0x7fa0b0101570 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0a6ffd700 1 -- 172.21.15.42:0/95767669 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000f620 con 0x7fa0b0101570 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.201+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b01a1e10 con 0x7fa0b0101570 2023-12-12T09:58:17.207 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.202+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b0068230 con 0x7fa0b0101570 2023-12-12T09:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.204+0000 7fa0a6ffd700 1 -- 172.21.15.42:0/95767669 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa0a0010460 con 0x7fa0b0101570 2023-12-12T09:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.204+0000 7fa0a6ffd700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa09c0a1590 0x7fa09c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.205+0000 7fa0b5ed8700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa09c0a1590 0x7fa09c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.205+0000 7fa0a6ffd700 1 -- 172.21.15.42:0/95767669 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa0a00cd040 con 0x7fa0b0101570 2023-12-12T09:58:17.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.205+0000 7fa0b5ed8700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa09c0a1590 0x7fa09c0a3a50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac007aa0 tx=0x7fa0ac00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:17.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.208+0000 7fa0a6ffd700 1 -- 172.21.15.42:0/95767669 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa0a0096230 con 0x7fa0b0101570 2023-12-12T09:58:17.416 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.413+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0b010bd10 con 0x7fa0b0101570 2023-12-12T09:58:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.415+0000 7fa0a6ffd700 1 -- 172.21.15.42:0/95767669 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24476 (secure 0 0 0) 0x7fa0a0020090 con 0x7fa0b0101570 2023-12-12T09:58:17.418 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:58:17.418 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:57:29.362715+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:58:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa09c0a1590 msgr2=0x7fa09c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa09c0a1590 0x7fa09c0a3a50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac007aa0 tx=0x7fa0ac00b040 comp rx=0 tx=0).stop 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0101570 msgr2=0x7fa0b0102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0101570 0x7fa0b0102950 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa0a000d950 tx=0x7fa0a000dd10 comp rx=0 tx=0).stop 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 shutdown_connections 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa09c0a1590 0x7fa09c0a3a50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0b0100380 0x7fa0b01042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0b0101f10 0x7fa0b0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 --2- 172.21.15.42:0/95767669 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0b0101570 0x7fa0b0102950 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 >> 172.21.15.42:0/95767669 conn(0x7fa0b00fb8f0 msgr2=0x7fa0b0068680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 shutdown_connections 2023-12-12T09:58:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.417+0000 7fa0b813c700 1 -- 172.21.15.42:0/95767669 wait complete. 2023-12-12T09:58:17.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:56:35.884237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:56:37.283813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:56:40.918865+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:58:17.433 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:58:17.565 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.561+0000 7f43f96ae700 1 -- 172.21.15.42:0/1166301138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 msgr2=0x7f43f41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.561+0000 7f43f96ae700 1 --2- 172.21.15.42:0/1166301138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 0x7f43f41088c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00b0d0 tx=0x7f43ec00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.561+0000 7f43f96ae700 1 -- 172.21.15.42:0/1166301138 shutdown_connections 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.561+0000 7f43f96ae700 1 --2- 172.21.15.42:0/1166301138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 0x7f43f41088c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.561+0000 7f43f96ae700 1 --2- 172.21.15.42:0/1166301138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f4100480 0x7f43f41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.561+0000 7f43f96ae700 1 --2- 172.21.15.42:0/1166301138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f4101670 0x7f43f4101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.561+0000 7f43f96ae700 1 -- 172.21.15.42:0/1166301138 >> 172.21.15.42:0/1166301138 conn(0x7f43f40fb9f0 msgr2=0x7f43f40fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.561+0000 7f43f96ae700 1 -- 172.21.15.42:0/1166301138 shutdown_connections 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.561+0000 7f43f96ae700 1 -- 172.21.15.42:0/1166301138 wait complete. 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f96ae700 1 Processor -- start 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f96ae700 1 -- start start 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f96ae700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f4100480 0x7f43f4069150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f96ae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f4101670 0x7f43f419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f96ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 0x7f43f419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.566 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f96ae700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f419f160 con 0x7f43f4102010 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f96ae700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f419f2d0 con 0x7f43f4100480 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f96ae700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f419f490 con 0x7f43f4101670 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 0x7f43f419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 0x7f43f419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34982/0 (socket says 172.21.15.42:34982) 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.562+0000 7f43f37fe700 1 -- 172.21.15.42:0/3385399315 learned_addr learned my addr 172.21.15.42:0/3385399315 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f2ffd700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f4100480 0x7f43f4069150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f27fc700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f4101670 0x7f43f419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f37fe700 1 -- 172.21.15.42:0/3385399315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f4101670 msgr2=0x7f43f419c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f37fe700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f4101670 0x7f43f419c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f37fe700 1 -- 172.21.15.42:0/3385399315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f4100480 msgr2=0x7f43f4069150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f37fe700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f4100480 0x7f43f4069150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f37fe700 1 -- 172.21.15.42:0/3385399315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ec009d00 con 0x7f43f4102010 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f37fe700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 0x7f43f419e970 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f43ec007b60 tx=0x7f43ec008f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43dbfff700 1 -- 172.21.15.42:0/3385399315 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec00e040 con 0x7f43f4102010 2023-12-12T09:58:17.567 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43dbfff700 1 -- 172.21.15.42:0/3385399315 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43ec00be70 con 0x7f43f4102010 2023-12-12T09:58:17.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f419f6b0 con 0x7f43f4102010 2023-12-12T09:58:17.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43dbfff700 1 -- 172.21.15.42:0/3385399315 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec01bb00 con 0x7f43f4102010 2023-12-12T09:58:17.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.563+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f419fae0 con 0x7f43f4102010 2023-12-12T09:58:17.568 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.564+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f4107830 con 0x7f43f4102010 2023-12-12T09:58:17.569 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.565+0000 7f43dbfff700 1 -- 172.21.15.42:0/3385399315 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f43ec01d670 con 0x7f43f4102010 2023-12-12T09:58:17.569 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.566+0000 7f43dbfff700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f43dc0a1510 0x7f43dc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.569 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.566+0000 7f43f2ffd700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f43dc0a1510 0x7f43dc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.569 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.566+0000 7f43dbfff700 1 -- 172.21.15.42:0/3385399315 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f43ec0cf090 con 0x7f43f4102010 2023-12-12T09:58:17.569 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.566+0000 7f43f2ffd700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f43dc0a1510 0x7f43dc0a39d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005950 tx=0x7f43e40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:17.571 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.569+0000 7f43dbfff700 1 -- 172.21.15.42:0/3385399315 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f43ec0982b0 con 0x7f43f4102010 2023-12-12T09:58:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:17 smithi111 ceph-mon[102565]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:17 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/205901271' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-12T09:58:17.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:17 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/95767669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:58:17.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.771+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43f410bce0 con 0x7f43f4102010 2023-12-12T09:58:17.773 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.771+0000 7f43dbfff700 1 -- 172.21.15.42:0/3385399315 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43ec09b8d0 con 0x7f43f4102010 2023-12-12T09:58:17.773 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f43dc0a1510 msgr2=0x7f43dc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f43dc0a1510 0x7f43dc0a39d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f43e4005950 tx=0x7f43e40058e0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 msgr2=0x7f43f419e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 0x7f43f419e970 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f43ec007b60 tx=0x7f43ec008f40 comp rx=0 tx=0).stop 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 shutdown_connections 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f43dc0a1510 0x7f43dc0a39d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f43f4100480 0x7f43f4069150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f43f4102010 0x7f43f419e970 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 --2- 172.21.15.42:0/3385399315 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f43f4101670 0x7f43f419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.773+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 >> 172.21.15.42:0/3385399315 conn(0x7f43f40fb9f0 msgr2=0x7f43f4104970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.774+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 shutdown_connections 2023-12-12T09:58:17.776 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.774+0000 7f43f96ae700 1 -- 172.21.15.42:0/3385399315 wait complete. 2023-12-12T09:58:17.787 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:17 smithi042 ceph-mon[98894]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:17.787 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:17 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/205901271' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-12T09:58:17.787 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:17 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/95767669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:58:17.788 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:58:17.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:17 smithi173 ceph-mon[103308]: pgmap v313: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:17.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:17 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/205901271' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-12T09:58:17.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:17 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/95767669' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:58:17.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 -- 172.21.15.42:0/374041334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 msgr2=0x7f0234106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 --2- 172.21.15.42:0/374041334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 0x7f0234106d90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0224009990 tx=0x7f0224009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 -- 172.21.15.42:0/374041334 shutdown_connections 2023-12-12T09:58:17.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 --2- 172.21.15.42:0/374041334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02341072d0 0x7f02341096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 --2- 172.21.15.42:0/374041334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0234102070 0x7f0234104460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 --2- 172.21.15.42:0/374041334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 0x7f0234106d90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 -- 172.21.15.42:0/374041334 >> 172.21.15.42:0/374041334 conn(0x7f02340fb9e0 msgr2=0x7f02340fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:17.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 -- 172.21.15.42:0/374041334 shutdown_connections 2023-12-12T09:58:17.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 -- 172.21.15.42:0/374041334 wait complete. 2023-12-12T09:58:17.918 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.915+0000 7f02396eb700 1 Processor -- start 2023-12-12T09:58:17.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02396eb700 1 -- start start 2023-12-12T09:58:17.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02396eb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0234102070 0x7f0234197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02396eb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 0x7f0234197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02396eb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02341072d0 0x7f023419e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.920 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02396eb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023419ef80 con 0x7f02341049a0 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02396eb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023419f0c0 con 0x7f0234102070 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02396eb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023419f250 con 0x7f02341072d0 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02327fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 0x7f0234197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02327fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 0x7f0234197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34994/0 (socket says 172.21.15.42:34994) 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f0232ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0234102070 0x7f0234197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02327fc700 1 -- 172.21.15.42:0/1865005476 learned_addr learned my addr 172.21.15.42:0/1865005476 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02327fc700 1 -- 172.21.15.42:0/1865005476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02341072d0 msgr2=0x7f023419e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02337fe700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02341072d0 0x7f023419e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02327fc700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02341072d0 0x7f023419e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02327fc700 1 -- 172.21.15.42:0/1865005476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0234102070 msgr2=0x7f0234197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02327fc700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0234102070 0x7f0234197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02327fc700 1 -- 172.21.15.42:0/1865005476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02240095f0 con 0x7f02341049a0 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f0232ffd700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0234102070 0x7f0234197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:58:17.921 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.916+0000 7f02327fc700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 0x7f0234197990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f02240095c0 tx=0x7f022400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:17.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.917+0000 7f022bfff700 1 -- 172.21.15.42:0/1865005476 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022401d070 con 0x7f02341049a0 2023-12-12T09:58:17.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.917+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023419f4d0 con 0x7f02341049a0 2023-12-12T09:58:17.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.917+0000 7f022bfff700 1 -- 172.21.15.42:0/1865005476 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0224005510 con 0x7f02341049a0 2023-12-12T09:58:17.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.917+0000 7f022bfff700 1 -- 172.21.15.42:0/1865005476 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224004c20 con 0x7f02341049a0 2023-12-12T09:58:17.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.917+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023419f950 con 0x7f02341049a0 2023-12-12T09:58:17.922 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.918+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02340fd560 con 0x7f02341049a0 2023-12-12T09:58:17.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.919+0000 7f022bfff700 1 -- 172.21.15.42:0/1865005476 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f022400fb90 con 0x7f02341049a0 2023-12-12T09:58:17.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.919+0000 7f022bfff700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f02200a1510 0x7f02200a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:17.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.920+0000 7f022bfff700 1 -- 172.21.15.42:0/1865005476 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f02240ce140 con 0x7f02341049a0 2023-12-12T09:58:17.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.920+0000 7f0232ffd700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f02200a1510 0x7f02200a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:17.923 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.920+0000 7f0232ffd700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f02200a1510 0x7f02200a39d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f021c00afd0 tx=0x7f021c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:17.926 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:17.924+0000 7f022bfff700 1 -- 172.21.15.42:0/1865005476 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f02240973c0 con 0x7f02341049a0 2023-12-12T09:58:18.125 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.123+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f023410cec0 con 0x7f02341049a0 2023-12-12T09:58:18.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.124+0000 7f022bfff700 1 -- 172.21.15.42:0/1865005476 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f022409a9e0 con 0x7f02341049a0 2023-12-12T09:58:18.126 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:58:18.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f02200a1510 msgr2=0x7f02200a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f02200a1510 0x7f02200a39d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f021c00afd0 tx=0x7f021c00a380 comp rx=0 tx=0).stop 2023-12-12T09:58:18.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 msgr2=0x7f0234197990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 0x7f0234197990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f02240095c0 tx=0x7f022400f690 comp rx=0 tx=0).stop 2023-12-12T09:58:18.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 shutdown_connections 2023-12-12T09:58:18.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f02200a1510 0x7f02200a39d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0234102070 0x7f0234197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f02341072d0 0x7f023419e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 --2- 172.21.15.42:0/1865005476 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02341049a0 0x7f0234197990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 >> 172.21.15.42:0/1865005476 conn(0x7f02340fb9e0 msgr2=0x7f0234102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:18.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 shutdown_connections 2023-12-12T09:58:18.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.126+0000 7f02396eb700 1 -- 172.21.15.42:0/1865005476 wait complete. 2023-12-12T09:58:18.141 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:58:18.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.267+0000 7f4a991a2700 1 -- 172.21.15.42:0/1955599768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 msgr2=0x7f4a941008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.267+0000 7f4a991a2700 1 --2- 172.21.15.42:0/1955599768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 0x7f4a941008a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4a840099c0 tx=0x7f4a84009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.268+0000 7f4a991a2700 1 -- 172.21.15.42:0/1955599768 shutdown_connections 2023-12-12T09:58:18.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.268+0000 7f4a991a2700 1 --2- 172.21.15.42:0/1955599768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a94102010 0x7f4a941088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.268+0000 7f4a991a2700 1 --2- 172.21.15.42:0/1955599768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 0x7f4a941008a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.268+0000 7f4a991a2700 1 --2- 172.21.15.42:0/1955599768 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a94101670 0x7f4a94101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.268+0000 7f4a991a2700 1 -- 172.21.15.42:0/1955599768 >> 172.21.15.42:0/1955599768 conn(0x7f4a940fb9f0 msgr2=0x7f4a940fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:18.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.268+0000 7f4a991a2700 1 -- 172.21.15.42:0/1955599768 shutdown_connections 2023-12-12T09:58:18.273 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.268+0000 7f4a991a2700 1 -- 172.21.15.42:0/1955599768 wait complete. 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.268+0000 7f4a991a2700 1 Processor -- start 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.268+0000 7f4a991a2700 1 -- start start 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a991a2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 0x7f4a94197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a991a2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a94101670 0x7f4a94197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a991a2700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a94102010 0x7f4a9419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a991a2700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9419f040 con 0x7f4a94100480 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a991a2700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9419f180 con 0x7f4a94102010 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a991a2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9419f310 con 0x7f4a94101670 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a92d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 0x7f4a94197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a92d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 0x7f4a94197510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35006/0 (socket says 172.21.15.42:35006) 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a92d9d700 1 -- 172.21.15.42:0/441732494 learned_addr learned my addr 172.21.15.42:0/441732494 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a9259c700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a94101670 0x7f4a94197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:18.274 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a9359e700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a94102010 0x7f4a9419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a92d9d700 1 -- 172.21.15.42:0/441732494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a94101670 msgr2=0x7f4a94197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a92d9d700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a94101670 0x7f4a94197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a92d9d700 1 -- 172.21.15.42:0/441732494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a94102010 msgr2=0x7f4a9419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a92d9d700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a94102010 0x7f4a9419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a92d9d700 1 -- 172.21.15.42:0/441732494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840095f0 con 0x7f4a94100480 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a9259c700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a94101670 0x7f4a94197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a92d9d700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 0x7f4a94197510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4a84003820 tx=0x7f4a84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.269+0000 7f4a7bfff700 1 -- 172.21.15.42:0/441732494 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8401d070 con 0x7f4a94100480 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.270+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9419f590 con 0x7f4a94100480 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.270+0000 7f4a7bfff700 1 -- 172.21.15.42:0/441732494 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a840054f0 con 0x7f4a94100480 2023-12-12T09:58:18.275 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.270+0000 7f4a7bfff700 1 -- 172.21.15.42:0/441732494 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a84004a00 con 0x7f4a94100480 2023-12-12T09:58:18.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.270+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9419fa40 con 0x7f4a94100480 2023-12-12T09:58:18.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.270+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a94068310 con 0x7f4a94100480 2023-12-12T09:58:18.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.272+0000 7f4a7bfff700 1 -- 172.21.15.42:0/441732494 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f4a84003b50 con 0x7f4a94100480 2023-12-12T09:58:18.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.272+0000 7f4a7bfff700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f4a7c0a1510 0x7f4a7c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:18.277 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.272+0000 7f4a7bfff700 1 -- 172.21.15.42:0/441732494 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4a840cdff0 con 0x7f4a94100480 2023-12-12T09:58:18.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.272+0000 7f4a9259c700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f4a7c0a1510 0x7f4a7c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:18.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.273+0000 7f4a9259c700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f4a7c0a1510 0x7f4a7c0a39d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a88005950 tx=0x7f4a880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:18.278 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.276+0000 7f4a7bfff700 1 -- 172.21.15.42:0/441732494 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4a84097270 con 0x7f4a94100480 2023-12-12T09:58:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.491+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a9419fd60 con 0x7f4a94100480 2023-12-12T09:58:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.492+0000 7f4a7bfff700 1 -- 172.21.15.42:0/441732494 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a8409a890 con 0x7f4a94100480 2023-12-12T09:58:18.494 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2023-12-12T09:58:18.496 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f4a7c0a1510 msgr2=0x7f4a7c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.496 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f4a7c0a1510 0x7f4a7c0a39d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4a88005950 tx=0x7f4a880058e0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 msgr2=0x7f4a94197510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 0x7f4a94197510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4a84003820 tx=0x7f4a84004730 comp rx=0 tx=0).stop 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 shutdown_connections 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f4a7c0a1510 0x7f4a7c0a39d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a94102010 0x7f4a9419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a94100480 0x7f4a94197510 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 --2- 172.21.15.42:0/441732494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a94101670 0x7f4a94197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 >> 172.21.15.42:0/441732494 conn(0x7f4a940fb9f0 msgr2=0x7f4a94068640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 shutdown_connections 2023-12-12T09:58:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.494+0000 7f4a991a2700 1 -- 172.21.15.42:0/441732494 wait complete. 2023-12-12T09:58:18.510 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:58:18.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.638+0000 7f6f8893b700 1 -- 172.21.15.42:0/3809571284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84100480 msgr2=0x7f6f841008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.642 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.638+0000 7f6f8893b700 1 --2- 172.21.15.42:0/3809571284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84100480 0x7f6f841008a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6f740099c0 tx=0x7f6f74009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.638+0000 7f6f8893b700 1 -- 172.21.15.42:0/3809571284 shutdown_connections 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.638+0000 7f6f8893b700 1 --2- 172.21.15.42:0/3809571284 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f84102010 0x7f6f841088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.638+0000 7f6f8893b700 1 --2- 172.21.15.42:0/3809571284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84100480 0x7f6f841008a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.638+0000 7f6f8893b700 1 --2- 172.21.15.42:0/3809571284 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f84101670 0x7f6f84101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.638+0000 7f6f8893b700 1 -- 172.21.15.42:0/3809571284 >> 172.21.15.42:0/3809571284 conn(0x7f6f840fb9f0 msgr2=0x7f6f840fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.638+0000 7f6f8893b700 1 -- 172.21.15.42:0/3809571284 shutdown_connections 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.638+0000 7f6f8893b700 1 -- 172.21.15.42:0/3809571284 wait complete. 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8893b700 1 Processor -- start 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8893b700 1 -- start start 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8893b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f84100480 0x7f6f84196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8893b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f84101670 0x7f6f84196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8893b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84102010 0x7f6f8419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8893b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8419df50 con 0x7f6f84102010 2023-12-12T09:58:18.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8893b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8419e090 con 0x7f6f84100480 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8893b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8419e220 con 0x7f6f84101670 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f82d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84102010 0x7f6f8419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f82d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84102010 0x7f6f8419d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35022/0 (socket says 172.21.15.42:35022) 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f82d9d700 1 -- 172.21.15.42:0/2062777843 learned_addr learned my addr 172.21.15.42:0/2062777843 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8259c700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f84100480 0x7f6f84196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f81d9b700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f84101670 0x7f6f84196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f82d9d700 1 -- 172.21.15.42:0/2062777843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f84101670 msgr2=0x7f6f84196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f82d9d700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f84101670 0x7f6f84196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f82d9d700 1 -- 172.21.15.42:0/2062777843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f84100480 msgr2=0x7f6f84196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f82d9d700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f84100480 0x7f6f84196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f82d9d700 1 -- 172.21.15.42:0/2062777843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f740095f0 con 0x7f6f84102010 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f8259c700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f84100480 0x7f6f84196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.639+0000 7f6f82d9d700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84102010 0x7f6f8419d980 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007ee0 tx=0x7f6f7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.640+0000 7f6f737fe700 1 -- 172.21.15.42:0/2062777843 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c017070 con 0x7f6f84102010 2023-12-12T09:58:18.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.640+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f8419e500 con 0x7f6f84102010 2023-12-12T09:58:18.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.640+0000 7f6f737fe700 1 -- 172.21.15.42:0/2062777843 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f7c00f040 con 0x7f6f84102010 2023-12-12T09:58:18.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.640+0000 7f6f737fe700 1 -- 172.21.15.42:0/2062777843 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c005740 con 0x7f6f84102010 2023-12-12T09:58:18.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.640+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f8419e9c0 con 0x7f6f84102010 2023-12-12T09:58:18.646 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.641+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f84107920 con 0x7f6f84102010 2023-12-12T09:58:18.646 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.642+0000 7f6f737fe700 1 -- 172.21.15.42:0/2062777843 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f6f7c0058a0 con 0x7f6f84102010 2023-12-12T09:58:18.646 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.642+0000 7f6f737fe700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f6c0a1510 0x7f6f6c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:18.646 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.643+0000 7f6f8259c700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f6c0a1510 0x7f6f6c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:18.646 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.643+0000 7f6f737fe700 1 -- 172.21.15.42:0/2062777843 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6f7c0cca70 con 0x7f6f84102010 2023-12-12T09:58:18.646 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.643+0000 7f6f8259c700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f6c0a1510 0x7f6f6c0a39d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400b580 tx=0x7f6f74005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:18.649 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.647+0000 7f6f737fe700 1 -- 172.21.15.42:0/2062777843 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6f7c095c60 con 0x7f6f84102010 2023-12-12T09:58:18.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:18 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3385399315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:58:18.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:18 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1865005476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:58:18.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:18 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3385399315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:58:18.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:18 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1865005476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:58:18.854 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.852+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6f84107300 con 0x7f6f84102010 2023-12-12T09:58:18.855 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.853+0000 7f6f737fe700 1 -- 172.21.15.42:0/2062777843 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6f7c099280 con 0x7f6f84102010 2023-12-12T09:58:18.855 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:58:18.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f6c0a1510 msgr2=0x7f6f6c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f6c0a1510 0x7f6f6c0a39d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6f7400b580 tx=0x7f6f74005fb0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84102010 msgr2=0x7f6f8419d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84102010 0x7f6f8419d980 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c007ee0 tx=0x7f6f7c00d3b0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 shutdown_connections 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f6f6c0a1510 0x7f6f6c0a39d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6f84100480 0x7f6f84196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6f84102010 0x7f6f8419d980 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 --2- 172.21.15.42:0/2062777843 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f84101670 0x7f6f84196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 >> 172.21.15.42:0/2062777843 conn(0x7f6f840fb9f0 msgr2=0x7f6f84104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 shutdown_connections 2023-12-12T09:58:18.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.855+0000 7f6f8893b700 1 -- 172.21.15.42:0/2062777843 wait complete. 2023-12-12T09:58:18.869 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:18 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3385399315' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:58:18.869 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:18 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1865005476' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:58:18.870 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:58:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.993+0000 7fb35575a700 1 -- 172.21.15.42:0/712440462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3501003b0 msgr2=0x7fb3501007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.993+0000 7fb35575a700 1 --2- 172.21.15.42:0/712440462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3501003b0 0x7fb3501007d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fb3400099c0 tx=0x7fb340009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.993+0000 7fb35575a700 1 -- 172.21.15.42:0/712440462 shutdown_connections 2023-12-12T09:58:18.999 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.993+0000 7fb35575a700 1 --2- 172.21.15.42:0/712440462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb350101f40 0x7fb3501087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.993+0000 7fb35575a700 1 --2- 172.21.15.42:0/712440462 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3501003b0 0x7fb3501007d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.993+0000 7fb35575a700 1 --2- 172.21.15.42:0/712440462 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3501015a0 0x7fb350101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.993+0000 7fb35575a700 1 -- 172.21.15.42:0/712440462 >> 172.21.15.42:0/712440462 conn(0x7fb3500fb920 msgr2=0x7fb3500fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.993+0000 7fb35575a700 1 -- 172.21.15.42:0/712440462 shutdown_connections 2023-12-12T09:58:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.993+0000 7fb35575a700 1 -- 172.21.15.42:0/712440462 wait complete. 2023-12-12T09:58:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.994+0000 7fb35575a700 1 Processor -- start 2023-12-12T09:58:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.994+0000 7fb35575a700 1 -- start start 2023-12-12T09:58:19.000 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.994+0000 7fb35575a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3501003b0 0x7fb35006a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.001 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.994+0000 7fb35575a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3501015a0 0x7fb35019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.001 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.994+0000 7fb35575a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb350101f40 0x7fb35019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.001 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.994+0000 7fb35575a700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35019eee0 con 0x7fb350101f40 2023-12-12T09:58:19.001 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.994+0000 7fb35575a700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35019f050 con 0x7fb3501003b0 2023-12-12T09:58:19.001 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.994+0000 7fb35575a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35019f210 con 0x7fb3501015a0 2023-12-12T09:58:19.001 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb350101f40 0x7fb35019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.001 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb350101f40 0x7fb35019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35036/0 (socket says 172.21.15.42:35036) 2023-12-12T09:58:19.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34f7fe700 1 -- 172.21.15.42:0/2172006065 learned_addr learned my addr 172.21.15.42:0/2172006065 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:19.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34e7fc700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3501015a0 0x7fb35019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34effd700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3501003b0 0x7fb35006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34f7fe700 1 -- 172.21.15.42:0/2172006065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3501015a0 msgr2=0x7fb35019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34f7fe700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3501015a0 0x7fb35019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.002 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34f7fe700 1 -- 172.21.15.42:0/2172006065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3501003b0 msgr2=0x7fb35006a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34f7fe700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3501003b0 0x7fb35006a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34f7fe700 1 -- 172.21.15.42:0/2172006065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3400095f0 con 0x7fb350101f40 2023-12-12T09:58:19.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb34f7fe700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb350101f40 0x7fb35019e740 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb348007ee0 tx=0x7fb34800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:19.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb337fff700 1 -- 172.21.15.42:0/2172006065 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348017070 con 0x7fb350101f40 2023-12-12T09:58:19.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb337fff700 1 -- 172.21.15.42:0/2172006065 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb34800f040 con 0x7fb350101f40 2023-12-12T09:58:19.003 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35019f4f0 con 0x7fb350101f40 2023-12-12T09:58:19.004 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb337fff700 1 -- 172.21.15.42:0/2172006065 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348005580 con 0x7fb350101f40 2023-12-12T09:58:19.004 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.995+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35019f9b0 con 0x7fb350101f40 2023-12-12T09:58:19.004 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.996+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3501077d0 con 0x7fb350101f40 2023-12-12T09:58:19.006 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.997+0000 7fb337fff700 1 -- 172.21.15.42:0/2172006065 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fb34800db20 con 0x7fb350101f40 2023-12-12T09:58:19.006 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.998+0000 7fb337fff700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb3380a1510 0x7fb3380a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.006 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.998+0000 7fb34effd700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb3380a1510 0x7fb3380a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.006 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.998+0000 7fb337fff700 1 -- 172.21.15.42:0/2172006065 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb3480ccf70 con 0x7fb350101f40 2023-12-12T09:58:19.006 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:18.999+0000 7fb34effd700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb3380a1510 0x7fb3380a39d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb340003820 tx=0x7fb340005920 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:19.006 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.002+0000 7fb337fff700 1 -- 172.21.15.42:0/2172006065 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fb348096160 con 0x7fb350101f40 2023-12-12T09:58:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.248+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb350068680 con 0x7fb350101f40 2023-12-12T09:58:19.251 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.249+0000 7fb337fff700 1 -- 172.21.15.42:0/2172006065 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11381 (secure 0 0 0) 0x7fb348099780 con 0x7fb350101f40 2023-12-12T09:58:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:58:19.251 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T09:57:07.293702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24487,"mds_3":24482,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:58:19.254 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb3380a1510 msgr2=0x7fb3380a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb3380a1510 0x7fb3380a39d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb340003820 tx=0x7fb340005920 comp rx=0 tx=0).stop 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb350101f40 msgr2=0x7fb35019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb350101f40 0x7fb35019e740 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb348007ee0 tx=0x7fb34800d3b0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 shutdown_connections 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fb3380a1510 0x7fb3380a39d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3501003b0 0x7fb35006a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb350101f40 0x7fb35019e740 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 --2- 172.21.15.42:0/2172006065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3501015a0 0x7fb35019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 >> 172.21.15.42:0/2172006065 conn(0x7fb3500fb920 msgr2=0x7fb350104910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 shutdown_connections 2023-12-12T09:58:19.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.252+0000 7fb35575a700 1 -- 172.21.15.42:0/2172006065 wait complete. 2023-12-12T09:58:19.256 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2023-12-12T09:58:19.268 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.393+0000 7fcaa4851700 1 -- 172.21.15.42:0/820476346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101f10 msgr2=0x7fcaa01087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.393+0000 7fcaa4851700 1 --2- 172.21.15.42:0/820476346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101f10 0x7fcaa01087c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fca9800b0d0 tx=0x7fca9800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.393+0000 7fcaa4851700 1 -- 172.21.15.42:0/820476346 shutdown_connections 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.393+0000 7fcaa4851700 1 --2- 172.21.15.42:0/820476346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101f10 0x7fcaa01087c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.393+0000 7fcaa4851700 1 --2- 172.21.15.42:0/820476346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaa0100380 0x7fcaa01007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.393+0000 7fcaa4851700 1 --2- 172.21.15.42:0/820476346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa0101570 0x7fcaa01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.393+0000 7fcaa4851700 1 -- 172.21.15.42:0/820476346 >> 172.21.15.42:0/820476346 conn(0x7fcaa00fb8f0 msgr2=0x7fcaa00fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.393+0000 7fcaa4851700 1 -- 172.21.15.42:0/820476346 shutdown_connections 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.393+0000 7fcaa4851700 1 -- 172.21.15.42:0/820476346 wait complete. 2023-12-12T09:58:19.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fcaa4851700 1 Processor -- start 2023-12-12T09:58:19.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fcaa4851700 1 -- start start 2023-12-12T09:58:19.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fcaa4851700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaa0100380 0x7fcaa01974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fcaa4851700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101570 0x7fcaa0197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fcaa4851700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa0101f10 0x7fcaa019ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fcaa4851700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa019f000 con 0x7fcaa0101570 2023-12-12T09:58:19.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fcaa4851700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa019f140 con 0x7fcaa0101f10 2023-12-12T09:58:19.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fcaa4851700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa019f2d0 con 0x7fcaa0100380 2023-12-12T09:58:19.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101570 0x7fcaa0197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9dd9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101570 0x7fcaa0197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35048/0 (socket says 172.21.15.42:35048) 2023-12-12T09:58:19.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9dd9b700 1 -- 172.21.15.42:0/3137634713 learned_addr learned my addr 172.21.15.42:0/3137634713 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:19.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9e59c700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaa0100380 0x7fcaa01974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9ed9d700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa0101f10 0x7fcaa019ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9dd9b700 1 -- 172.21.15.42:0/3137634713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaa0100380 msgr2=0x7fcaa01974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9dd9b700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaa0100380 0x7fcaa01974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9dd9b700 1 -- 172.21.15.42:0/3137634713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa0101f10 msgr2=0x7fcaa019ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9dd9b700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa0101f10 0x7fcaa019ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9dd9b700 1 -- 172.21.15.42:0/3137634713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca98009d00 con 0x7fcaa0101570 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9ed9d700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa0101f10 0x7fcaa019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.394+0000 7fca9dd9b700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101570 0x7fcaa0197a10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d950 tx=0x7fca9400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.395+0000 7fca8f7fe700 1 -- 172.21.15.42:0/3137634713 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94009940 con 0x7fcaa0101570 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.395+0000 7fca8f7fe700 1 -- 172.21.15.42:0/3137634713 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca94010460 con 0x7fcaa0101570 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.395+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa019f5b0 con 0x7fcaa0101570 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.395+0000 7fca8f7fe700 1 -- 172.21.15.42:0/3137634713 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9400f620 con 0x7fcaa0101570 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.395+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa019fb10 con 0x7fcaa0101570 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.396+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa0068230 con 0x7fcaa0101570 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.397+0000 7fca8f7fe700 1 -- 172.21.15.42:0/3137634713 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fca940108d0 con 0x7fcaa0101570 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.397+0000 7fca8f7fe700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fca880a1590 0x7fca880a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.398+0000 7fca9e59c700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fca880a1590 0x7fca880a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.398+0000 7fca8f7fe700 1 -- 172.21.15.42:0/3137634713 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fca940cced0 con 0x7fcaa0101570 2023-12-12T09:58:19.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.398+0000 7fca9e59c700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fca880a1590 0x7fca880a3a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcaa01013d0 tx=0x7fca90007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:19.405 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.402+0000 7fca8f7fe700 1 -- 172.21.15.42:0/3137634713 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fca940960c0 con 0x7fcaa0101570 2023-12-12T09:58:19.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.608+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcaa010bdf0 con 0x7fcaa0101570 2023-12-12T09:58:19.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.610+0000 7fca8f7fe700 1 -- 172.21.15.42:0/3137634713 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24476 (secure 0 0 0) 0x7fca94016090 con 0x7fcaa0101570 2023-12-12T09:58:19.612 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:58:19.612 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:57:29.362715+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:58:19.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fca880a1590 msgr2=0x7fca880a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fca880a1590 0x7fca880a3a50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcaa01013d0 tx=0x7fca90007400 comp rx=0 tx=0).stop 2023-12-12T09:58:19.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101570 msgr2=0x7fcaa0197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101570 0x7fcaa0197a10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fca9400d950 tx=0x7fca9400dc60 comp rx=0 tx=0).stop 2023-12-12T09:58:19.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 shutdown_connections 2023-12-12T09:58:19.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fca880a1590 0x7fca880a3a50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcaa0101f10 0x7fcaa019ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcaa0100380 0x7fcaa01974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 --2- 172.21.15.42:0/3137634713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcaa0101570 0x7fcaa0197a10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 >> 172.21.15.42:0/3137634713 conn(0x7fcaa00fb8f0 msgr2=0x7fcaa0068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:19.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 shutdown_connections 2023-12-12T09:58:19.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.612+0000 7fcaa4851700 1 -- 172.21.15.42:0/3137634713 wait complete. 2023-12-12T09:58:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:19 smithi042 ceph-mon[98894]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:58:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:19 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/441732494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:58:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:19 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2062777843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:58:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:19 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2172006065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:58:19.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:56:35.884237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:56:37.283813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:56:40.918865+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T09:58:19.628 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T09:58:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:19 smithi111 ceph-mon[102565]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:58:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:19 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/441732494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:58:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:19 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2062777843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:58:19.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:19 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2172006065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:58:19.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.754+0000 7f83e6172700 1 -- 172.21.15.42:0/2462476563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01016e0 msgr2=0x7f83e0101b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.754+0000 7f83e6172700 1 --2- 172.21.15.42:0/2462476563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01016e0 0x7f83e0101b60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009990 tx=0x7f83c8009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.754+0000 7f83e6172700 1 -- 172.21.15.42:0/2462476563 shutdown_connections 2023-12-12T09:58:19.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.754+0000 7f83e6172700 1 --2- 172.21.15.42:0/2462476563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e01009d0 0x7f83e0100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.754+0000 7f83e6172700 1 --2- 172.21.15.42:0/2462476563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83e01020a0 0x7f83e0108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.754+0000 7f83e6172700 1 --2- 172.21.15.42:0/2462476563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01016e0 0x7f83e0101b60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.754+0000 7f83e6172700 1 -- 172.21.15.42:0/2462476563 >> 172.21.15.42:0/2462476563 conn(0x7f83e00fb1a0 msgr2=0x7f83e00fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:19.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.754+0000 7f83e6172700 1 -- 172.21.15.42:0/2462476563 shutdown_connections 2023-12-12T09:58:19.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.754+0000 7f83e6172700 1 -- 172.21.15.42:0/2462476563 wait complete. 2023-12-12T09:58:19.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.755+0000 7f83e6172700 1 Processor -- start 2023-12-12T09:58:19.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.755+0000 7f83e6172700 1 -- start start 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.755+0000 7f83e6172700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e01009d0 0x7f83e01975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.755+0000 7f83e6172700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83e01016e0 0x7f83e0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.755+0000 7f83e6172700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01020a0 0x7f83e019eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.755+0000 7f83e6172700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e019f100 con 0x7f83e01020a0 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.755+0000 7f83e6172700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e019f240 con 0x7f83e01009d0 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.755+0000 7f83e6172700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e019f3d0 con 0x7f83e01016e0 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.755+0000 7f83dffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01020a0 0x7f83e019eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01020a0 0x7f83e019eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35064/0 (socket says 172.21.15.42:35064) 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dffff700 1 -- 172.21.15.42:0/4149696073 learned_addr learned my addr 172.21.15.42:0/4149696073 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83deffd700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83e01016e0 0x7f83e0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83df7fe700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e01009d0 0x7f83e01975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.760 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dffff700 1 -- 172.21.15.42:0/4149696073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83e01016e0 msgr2=0x7f83e0197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dffff700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83e01016e0 0x7f83e0197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dffff700 1 -- 172.21.15.42:0/4149696073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e01009d0 msgr2=0x7f83e01975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dffff700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e01009d0 0x7f83e01975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dffff700 1 -- 172.21.15.42:0/4149696073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c80095f0 con 0x7f83e01020a0 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83df7fe700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e01009d0 0x7f83e01975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dffff700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01020a0 0x7f83e019eb30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f83d8007ee0 tx=0x7f83d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dcff9700 1 -- 172.21.15.42:0/4149696073 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8017070 con 0x7f83e01020a0 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dcff9700 1 -- 172.21.15.42:0/4149696073 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83d800f040 con 0x7f83e01020a0 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e019f6b0 con 0x7f83e01020a0 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83dcff9700 1 -- 172.21.15.42:0/4149696073 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8005740 con 0x7f83e01020a0 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.756+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e019fbc0 con 0x7f83e01020a0 2023-12-12T09:58:19.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.757+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e0068760 con 0x7f83e01020a0 2023-12-12T09:58:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.759+0000 7f83dcff9700 1 -- 172.21.15.42:0/4149696073 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f83d800dc60 con 0x7f83e01020a0 2023-12-12T09:58:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.759+0000 7f83dcff9700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83cc0a1510 0x7f83cc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.759+0000 7f83df7fe700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83cc0a1510 0x7f83cc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.759+0000 7f83dcff9700 1 -- 172.21.15.42:0/4149696073 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f83d80cd030 con 0x7f83e01020a0 2023-12-12T09:58:19.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.760+0000 7f83df7fe700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83cc0a1510 0x7f83cc0a39d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f83e0069680 tx=0x7f83d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:19.765 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.763+0000 7f83dcff9700 1 -- 172.21.15.42:0/4149696073 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f83d8096220 con 0x7f83e01020a0 2023-12-12T09:58:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:19 smithi173 ceph-mon[103308]: pgmap v314: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T09:58:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:19 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/441732494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:58:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:19 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2062777843' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:58:19.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:19 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2172006065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:58:19.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.965+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83e010bd40 con 0x7f83e01020a0 2023-12-12T09:58:19.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.965+0000 7f83dcff9700 1 -- 172.21.15.42:0/4149696073 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83d8099840 con 0x7f83e01020a0 2023-12-12T09:58:19.968 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.967+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83cc0a1510 msgr2=0x7f83cc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.967+0000 7f83e6172700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83cc0a1510 0x7f83cc0a39d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f83e0069680 tx=0x7f83d0009260 comp rx=0 tx=0).stop 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.967+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01020a0 msgr2=0x7f83e019eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.967+0000 7f83e6172700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01020a0 0x7f83e019eb30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f83d8007ee0 tx=0x7f83d800d3b0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.968+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 shutdown_connections 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.968+0000 7f83e6172700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f83cc0a1510 0x7f83cc0a39d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.968+0000 7f83e6172700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f83e01009d0 0x7f83e01975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.968+0000 7f83e6172700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f83e01020a0 0x7f83e019eb30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.968+0000 7f83e6172700 1 --2- 172.21.15.42:0/4149696073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f83e01016e0 0x7f83e0197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:19.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.968+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 >> 172.21.15.42:0/4149696073 conn(0x7f83e00fb1a0 msgr2=0x7f83e00ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.968+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 shutdown_connections 2023-12-12T09:58:19.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:19.968+0000 7f83e6172700 1 -- 172.21.15.42:0/4149696073 wait complete. 2023-12-12T09:58:19.982 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T09:58:20.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 -- 172.21.15.42:0/701649223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 msgr2=0x7f63201043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 --2- 172.21.15.42:0/701649223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 0x7f63201043f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6310009990 tx=0x7f6310009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 -- 172.21.15.42:0/701649223 shutdown_connections 2023-12-12T09:58:20.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 --2- 172.21.15.42:0/701649223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6320107260 0x7f63201096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 --2- 172.21.15.42:0/701649223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 0x7f63201043f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 --2- 172.21.15.42:0/701649223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6320104930 0x7f6320106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 -- 172.21.15.42:0/701649223 >> 172.21.15.42:0/701649223 conn(0x7f63200fb9f0 msgr2=0x7f63200fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 -- 172.21.15.42:0/701649223 shutdown_connections 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 -- 172.21.15.42:0/701649223 wait complete. 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 Processor -- start 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.110+0000 7f6326521700 1 -- start start 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f6326521700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 0x7f63201996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f6326521700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6320104930 0x7f6320199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f6326521700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6320107260 0x7f63201a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f6326521700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63201a1210 con 0x7f6320102000 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f6326521700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63201a1350 con 0x7f6320107260 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f6326521700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63201a14e0 con 0x7f6320104930 2023-12-12T09:58:20.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 0x7f63201996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 0x7f63201996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35074/0 (socket says 172.21.15.42:35074) 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631ffff700 1 -- 172.21.15.42:0/1506672541 learned_addr learned my addr 172.21.15.42:0/1506672541 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f6324abe700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6320107260 0x7f63201a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631f7fe700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6320104930 0x7f6320199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631ffff700 1 -- 172.21.15.42:0/1506672541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6320104930 msgr2=0x7f6320199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631ffff700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6320104930 0x7f6320199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631ffff700 1 -- 172.21.15.42:0/1506672541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6320107260 msgr2=0x7f63201a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631ffff700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6320107260 0x7f63201a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631ffff700 1 -- 172.21.15.42:0/1506672541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63100095f0 con 0x7f6320102000 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631f7fe700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6320104930 0x7f6320199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.111+0000 7f631ffff700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 0x7f63201996e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6310000c00 tx=0x7f6310004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.112+0000 7f631d7fa700 1 -- 172.21.15.42:0/1506672541 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f631001d070 con 0x7f6320102000 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.112+0000 7f631d7fa700 1 -- 172.21.15.42:0/1506672541 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63100054f0 con 0x7f6320102000 2023-12-12T09:58:20.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.112+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63201a1760 con 0x7f6320102000 2023-12-12T09:58:20.118 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.112+0000 7f631d7fa700 1 -- 172.21.15.42:0/1506672541 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310004cd0 con 0x7f6320102000 2023-12-12T09:58:20.118 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.112+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63201a1b20 con 0x7f6320102000 2023-12-12T09:58:20.118 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.113+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6320108e60 con 0x7f6320102000 2023-12-12T09:58:20.118 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.114+0000 7f631d7fa700 1 -- 172.21.15.42:0/1506672541 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f631000fc40 con 0x7f6320102000 2023-12-12T09:58:20.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.114+0000 7f631d7fa700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f630c0a1510 0x7f630c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.115+0000 7f631f7fe700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f630c0a1510 0x7f630c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.115+0000 7f631d7fa700 1 -- 172.21.15.42:0/1506672541 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f63100ce2b0 con 0x7f6320102000 2023-12-12T09:58:20.119 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.115+0000 7f631f7fe700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f630c0a1510 0x7f630c0a39d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6308005fd0 tx=0x7f6308005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:20.121 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.119+0000 7f631d7fa700 1 -- 172.21.15.42:0/1506672541 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f63100975d0 con 0x7f6320102000 2023-12-12T09:58:20.321 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.318+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f632010ceb0 con 0x7f6320102000 2023-12-12T09:58:20.321 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.319+0000 7f631d7fa700 1 -- 172.21.15.42:0/1506672541 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f631009abf0 con 0x7f6320102000 2023-12-12T09:58:20.321 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:58:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f630c0a1510 msgr2=0x7f630c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f630c0a1510 0x7f630c0a39d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6308005fd0 tx=0x7f6308005f60 comp rx=0 tx=0).stop 2023-12-12T09:58:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 msgr2=0x7f63201996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 0x7f63201996e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6310000c00 tx=0x7f6310004730 comp rx=0 tx=0).stop 2023-12-12T09:58:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 shutdown_connections 2023-12-12T09:58:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f630c0a1510 0x7f630c0a39d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6320107260 0x7f63201a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6320102000 0x7f63201996e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.324 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 --2- 172.21.15.42:0/1506672541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6320104930 0x7f6320199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.324 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 >> 172.21.15.42:0/1506672541 conn(0x7f63200fb9f0 msgr2=0x7f63201018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:20.324 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 shutdown_connections 2023-12-12T09:58:20.324 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.321+0000 7f6326521700 1 -- 172.21.15.42:0/1506672541 wait complete. 2023-12-12T09:58:20.335 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 -- 172.21.15.42:0/3767202569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 msgr2=0x7f28701088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 --2- 172.21.15.42:0/3767202569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 0x7f28701088c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f286800b0d0 tx=0x7f286800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 -- 172.21.15.42:0/3767202569 shutdown_connections 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 --2- 172.21.15.42:0/3767202569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 0x7f28701088c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 --2- 172.21.15.42:0/3767202569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2870100480 0x7f28701008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 --2- 172.21.15.42:0/3767202569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2870101670 0x7f2870101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 -- 172.21.15.42:0/3767202569 >> 172.21.15.42:0/3767202569 conn(0x7f28700fb9f0 msgr2=0x7f28700fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 -- 172.21.15.42:0/3767202569 shutdown_connections 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 -- 172.21.15.42:0/3767202569 wait complete. 2023-12-12T09:58:20.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.462+0000 7f2877f71700 1 Processor -- start 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2877f71700 1 -- start start 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2877f71700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2870100480 0x7f28701043c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2877f71700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2870101670 0x7f2870102a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2877f71700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 0x7f2870102f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2877f71700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2870103730 con 0x7f2870102010 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2877f71700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28701038a0 con 0x7f2870100480 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2877f71700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28701a16b0 con 0x7f2870101670 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287650e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 0x7f2870102f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287650e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 0x7f2870102f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35092/0 (socket says 172.21.15.42:35092) 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287650e700 1 -- 172.21.15.42:0/3460762989 learned_addr learned my addr 172.21.15.42:0/3460762989 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2875d0d700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2870100480 0x7f28701043c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287550c700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2870101670 0x7f2870102a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287650e700 1 -- 172.21.15.42:0/3460762989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2870101670 msgr2=0x7f2870102a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287650e700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2870101670 0x7f2870102a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287650e700 1 -- 172.21.15.42:0/3460762989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2870100480 msgr2=0x7f28701043c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287650e700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2870100480 0x7f28701043c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287650e700 1 -- 172.21.15.42:0/3460762989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2868009d00 con 0x7f2870102010 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287550c700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2870101670 0x7f2870102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f287650e700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 0x7f2870102f90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2868007bd0 tx=0x7f2868009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2866ffd700 1 -- 172.21.15.42:0/3460762989 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286800e040 con 0x7f2870102010 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.463+0000 7f2866ffd700 1 -- 172.21.15.42:0/3460762989 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2868012dc0 con 0x7f2870102010 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.464+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28701a18e0 con 0x7f2870102010 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.464+0000 7f2866ffd700 1 -- 172.21.15.42:0/3460762989 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286801b450 con 0x7f2870102010 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.464+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28701a1d00 con 0x7f2870102010 2023-12-12T09:58:20.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.465+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2870068310 con 0x7f2870102010 2023-12-12T09:58:20.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.466+0000 7f2866ffd700 1 -- 172.21.15.42:0/3460762989 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2868012900 con 0x7f2870102010 2023-12-12T09:58:20.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.466+0000 7f2866ffd700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f285c0a1590 0x7f285c0a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.467+0000 7f2875d0d700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f285c0a1590 0x7f285c0a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.467+0000 7f2866ffd700 1 -- 172.21.15.42:0/3460762989 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f28680ceff0 con 0x7f2870102010 2023-12-12T09:58:20.471 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.467+0000 7f2875d0d700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f285c0a1590 0x7f285c0a3a50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f28701014d0 tx=0x7f286000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:20.473 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.471+0000 7f2866ffd700 1 -- 172.21.15.42:0/3460762989 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2868098270 con 0x7f2870102010 2023-12-12T09:58:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:20 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3137634713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:58:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:20 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/4149696073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:58:20.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:20 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1506672541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:58:20.675 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.673+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28701a2020 con 0x7f2870102010 2023-12-12T09:58:20.675 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.673+0000 7f2866ffd700 1 -- 172.21.15.42:0/3460762989 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f286809b890 con 0x7f2870102010 2023-12-12T09:58:20.675 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2023-12-12T09:58:20.677 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f285c0a1590 msgr2=0x7f285c0a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.677 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f285c0a1590 0x7f285c0a3a50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f28701014d0 tx=0x7f286000b410 comp rx=0 tx=0).stop 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 msgr2=0x7f2870102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 0x7f2870102f90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2868007bd0 tx=0x7f2868009440 comp rx=0 tx=0).stop 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 shutdown_connections 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f285c0a1590 0x7f285c0a3a50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2870100480 0x7f28701043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2870102010 0x7f2870102f90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 --2- 172.21.15.42:0/3460762989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2870101670 0x7f2870102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 >> 172.21.15.42:0/3460762989 conn(0x7f28700fb9f0 msgr2=0x7f28700fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 shutdown_connections 2023-12-12T09:58:20.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.675+0000 7f2877f71700 1 -- 172.21.15.42:0/3460762989 wait complete. 2023-12-12T09:58:20.691 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T09:58:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:20 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3137634713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:58:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:20 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/4149696073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:58:20.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:20 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1506672541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:58:20.820 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.818+0000 7fe009793700 1 -- 172.21.15.42:0/3058817313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 msgr2=0x7fe0041008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.818+0000 7fe009793700 1 --2- 172.21.15.42:0/3058817313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 0x7fe0041008d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdff80099c0 tx=0x7fdff8009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.818+0000 7fe009793700 1 -- 172.21.15.42:0/3058817313 shutdown_connections 2023-12-12T09:58:20.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.818+0000 7fe009793700 1 --2- 172.21.15.42:0/3058817313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe004102040 0x7fe0041088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.818+0000 7fe009793700 1 --2- 172.21.15.42:0/3058817313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 0x7fe0041008d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.818+0000 7fe009793700 1 --2- 172.21.15.42:0/3058817313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0041016a0 0x7fe004101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.818+0000 7fe009793700 1 -- 172.21.15.42:0/3058817313 >> 172.21.15.42:0/3058817313 conn(0x7fe0040fba20 msgr2=0x7fe0040fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:20.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.818+0000 7fe009793700 1 -- 172.21.15.42:0/3058817313 shutdown_connections 2023-12-12T09:58:20.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.818+0000 7fe009793700 1 -- 172.21.15.42:0/3058817313 wait complete. 2023-12-12T09:58:20.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe009793700 1 Processor -- start 2023-12-12T09:58:20.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe009793700 1 -- start start 2023-12-12T09:58:20.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe009793700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 0x7fe0041963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe009793700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0041016a0 0x7fe004196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe009793700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe004102040 0x7fe00419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe009793700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00419df10 con 0x7fe0041004b0 2023-12-12T09:58:20.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe009793700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00419e050 con 0x7fe004102040 2023-12-12T09:58:20.824 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe009793700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00419e1e0 con 0x7fe0041016a0 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe002ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 0x7fe0041963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe002ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 0x7fe0041963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35102/0 (socket says 172.21.15.42:35102) 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe002ffd700 1 -- 172.21.15.42:0/2176372157 learned_addr learned my addr 172.21.15.42:0/2176372157 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe0037fe700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe004102040 0x7fe00419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.819+0000 7fe0027fc700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0041016a0 0x7fe004196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fe002ffd700 1 -- 172.21.15.42:0/2176372157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0041016a0 msgr2=0x7fe004196920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fe002ffd700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0041016a0 0x7fe004196920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fe002ffd700 1 -- 172.21.15.42:0/2176372157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe004102040 msgr2=0x7fe00419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fe002ffd700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe004102040 0x7fe00419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fe002ffd700 1 -- 172.21.15.42:0/2176372157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff80095f0 con 0x7fe0041004b0 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fe0027fc700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0041016a0 0x7fe004196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:20.825 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fe002ffd700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 0x7fe0041963e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009fd0 tx=0x7fdff8005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fdfebfff700 1 -- 172.21.15.42:0/2176372157 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff801f070 con 0x7fe0041004b0 2023-12-12T09:58:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe004069ab0 con 0x7fe0041004b0 2023-12-12T09:58:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fdfebfff700 1 -- 172.21.15.42:0/2176372157 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdff80059a0 con 0x7fe0041004b0 2023-12-12T09:58:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fdfebfff700 1 -- 172.21.15.42:0/2176372157 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8025420 con 0x7fe0041004b0 2023-12-12T09:58:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.820+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe004069f60 con 0x7fe0041004b0 2023-12-12T09:58:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.821+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe004068760 con 0x7fe0041004b0 2023-12-12T09:58:20.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.822+0000 7fdfebfff700 1 -- 172.21.15.42:0/2176372157 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fdff8023070 con 0x7fe0041004b0 2023-12-12T09:58:20.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:20 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3137634713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:58:20.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:20 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/4149696073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T09:58:20.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:20 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1506672541' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T09:58:20.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.823+0000 7fdfebfff700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdfec0a1510 0x7fdfec0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:20.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.823+0000 7fe0027fc700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdfec0a1510 0x7fdfec0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:20.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.823+0000 7fdfebfff700 1 -- 172.21.15.42:0/2176372157 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdff80d08d0 con 0x7fe0041004b0 2023-12-12T09:58:20.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.823+0000 7fe0027fc700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdfec0a1510 0x7fdfec0a39d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009cb0 tx=0x7fdff4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:20.829 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:20.826+0000 7fdfebfff700 1 -- 172.21.15.42:0/2176372157 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fdff8099af0 con 0x7fe0041004b0 2023-12-12T09:58:21.033 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.031+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe00410bec0 con 0x7fe0041004b0 2023-12-12T09:58:21.034 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.032+0000 7fdfebfff700 1 -- 172.21.15.42:0/2176372157 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdff809d110 con 0x7fe0041004b0 2023-12-12T09:58:21.034 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T09:58:21.036 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdfec0a1510 msgr2=0x7fdfec0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.036 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdfec0a1510 0x7fdfec0a39d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdff4009cb0 tx=0x7fdff4009260 comp rx=0 tx=0).stop 2023-12-12T09:58:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 msgr2=0x7fe0041963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 0x7fe0041963e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009fd0 tx=0x7fdff8005f10 comp rx=0 tx=0).stop 2023-12-12T09:58:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 shutdown_connections 2023-12-12T09:58:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fdfec0a1510 0x7fdfec0a39d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe004102040 0x7fe00419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.037 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe0041004b0 0x7fe0041963e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.038 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 --2- 172.21.15.42:0/2176372157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0041016a0 0x7fe004196920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.038 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 >> 172.21.15.42:0/2176372157 conn(0x7fe0040fba20 msgr2=0x7fe004104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.038 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 shutdown_connections 2023-12-12T09:58:21.038 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.034+0000 7fe009793700 1 -- 172.21.15.42:0/2176372157 wait complete. 2023-12-12T09:58:21.049 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T09:58:21.177 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.173+0000 7f52e3411700 1 -- 172.21.15.42:0/1132436445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 msgr2=0x7f52dc1095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.177 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.173+0000 7f52e3411700 1 --2- 172.21.15.42:0/1132436445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 0x7f52dc1095d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f52d400b0a0 tx=0x7f52d400b3b0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.177 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.173+0000 7f52e3411700 1 -- 172.21.15.42:0/1132436445 shutdown_connections 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.173+0000 7f52e3411700 1 --2- 172.21.15.42:0/1132436445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 0x7f52dc1095d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.173+0000 7f52e3411700 1 --2- 172.21.15.42:0/1132436445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52dc101f60 0x7f52dc104350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.173+0000 7f52e3411700 1 --2- 172.21.15.42:0/1132436445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52dc104890 0x7f52dc106c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.173+0000 7f52e3411700 1 -- 172.21.15.42:0/1132436445 >> 172.21.15.42:0/1132436445 conn(0x7f52dc0fb8f0 msgr2=0x7f52dc0fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.173+0000 7f52e3411700 1 -- 172.21.15.42:0/1132436445 shutdown_connections 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.173+0000 7f52e3411700 1 -- 172.21.15.42:0/1132436445 wait complete. 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e3411700 1 Processor -- start 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e3411700 1 -- start start 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e3411700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52dc101f60 0x7f52dc1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e3411700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52dc104890 0x7f52dc1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e3411700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 0x7f52dc19e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e3411700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc19ef90 con 0x7f52dc1071c0 2023-12-12T09:58:21.178 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e3411700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc19f0d0 con 0x7f52dc104890 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e3411700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc19f260 con 0x7f52dc101f60 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e19ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 0x7f52dc19e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e19ae700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 0x7f52dc19e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35120/0 (socket says 172.21.15.42:35120) 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e19ae700 1 -- 172.21.15.42:0/4056095368 learned_addr learned my addr 172.21.15.42:0/4056095368 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e11ad700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52dc101f60 0x7f52dc1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e09ac700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52dc104890 0x7f52dc1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.174+0000 7f52e19ae700 1 -- 172.21.15.42:0/4056095368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52dc101f60 msgr2=0x7f52dc1974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52e19ae700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52dc101f60 0x7f52dc1974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52e19ae700 1 -- 172.21.15.42:0/4056095368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52dc104890 msgr2=0x7f52dc1979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52e19ae700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52dc104890 0x7f52dc1979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52e19ae700 1 -- 172.21.15.42:0/4056095368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d4009d00 con 0x7f52dc1071c0 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52e19ae700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 0x7f52dc19e9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f52d4007a60 tx=0x7f52d400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52d27fc700 1 -- 172.21.15.42:0/4056095368 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d400e040 con 0x7f52dc1071c0 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52d27fc700 1 -- 172.21.15.42:0/4056095368 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52d40094c0 con 0x7f52dc1071c0 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc0ffd50 con 0x7f52dc1071c0 2023-12-12T09:58:21.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52d27fc700 1 -- 172.21.15.42:0/4056095368 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52d40052a0 con 0x7f52dc1071c0 2023-12-12T09:58:21.180 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.175+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc100260 con 0x7f52dc1071c0 2023-12-12T09:58:21.180 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.176+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc0fd470 con 0x7f52dc1071c0 2023-12-12T09:58:21.180 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.177+0000 7f52d27fc700 1 -- 172.21.15.42:0/4056095368 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f52d401d470 con 0x7f52dc1071c0 2023-12-12T09:58:21.180 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.178+0000 7f52d27fc700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f52c80a1510 0x7f52c80a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.181 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.178+0000 7f52d27fc700 1 -- 172.21.15.42:0/4056095368 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f52d40ceaf0 con 0x7f52dc1071c0 2023-12-12T09:58:21.181 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.178+0000 7f52e11ad700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f52c80a1510 0x7f52c80a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.181 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.178+0000 7f52e11ad700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f52c80a1510 0x7f52c80a39d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009c60 tx=0x7f52cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.183 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.181+0000 7f52d27fc700 1 -- 172.21.15.42:0/4056095368 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f52d4097d10 con 0x7f52dc1071c0 2023-12-12T09:58:21.436 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.434+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52dc068230 con 0x7f52dc1071c0 2023-12-12T09:58:21.437 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.435+0000 7f52d27fc700 1 -- 172.21.15.42:0/4056095368 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f52d409b330 con 0x7f52dc1071c0 2023-12-12T09:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:58:21.437 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T09:57:07.293702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24487,"mds_3":24482,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T09:58:21.439 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f52c80a1510 msgr2=0x7f52c80a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.439 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f52c80a1510 0x7f52c80a39d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f52cc009c60 tx=0x7f52cc009260 comp rx=0 tx=0).stop 2023-12-12T09:58:21.439 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 msgr2=0x7f52dc19e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.439 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 0x7f52dc19e9c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f52d4007a60 tx=0x7f52d400bce0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.439 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 shutdown_connections 2023-12-12T09:58:21.439 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f52c80a1510 0x7f52c80a39d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.439 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52dc1071c0 0x7f52dc19e9c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.439 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52dc101f60 0x7f52dc1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.439 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 --2- 172.21.15.42:0/4056095368 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52dc104890 0x7f52dc1979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.440 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 >> 172.21.15.42:0/4056095368 conn(0x7f52dc0fb8f0 msgr2=0x7f52dc1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.440 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 shutdown_connections 2023-12-12T09:58:21.440 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.437+0000 7f52e3411700 1 -- 172.21.15.42:0/4056095368 wait complete. 2023-12-12T09:58:21.440 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 27 2023-12-12T09:58:21.452 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.577+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1641839265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b0100f80 msgr2=0x7fa1b0108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.577+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1641839265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b0100f80 0x7fa1b0108820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800b0a0 tx=0x7fa1a800b3b0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.577+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1641839265 shutdown_connections 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.577+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1641839265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b0101970 0x7fa1b0101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.577+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1641839265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b0100f80 0x7fa1b0108820 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.577+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1641839265 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1b01005c0 0x7fa1b0100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.577+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1641839265 >> 172.21.15.42:0/1641839265 conn(0x7fa1b00fb0b0 msgr2=0x7fa1b00fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.577+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1641839265 shutdown_connections 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.577+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1641839265 wait complete. 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.578+0000 7fa1b54a3700 1 Processor -- start 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.578+0000 7fa1b54a3700 1 -- start start 2023-12-12T09:58:21.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.578+0000 7fa1b54a3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b01005c0 0x7fa1b0197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.578+0000 7fa1b54a3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1b0100f80 0x7fa1b01979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.578+0000 7fa1b54a3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b0101970 0x7fa1b019e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.578+0000 7fa1b54a3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b019efb0 con 0x7fa1b01005c0 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.578+0000 7fa1b54a3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b019f0f0 con 0x7fa1b0101970 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.578+0000 7fa1b54a3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b019f280 con 0x7fa1b0100f80 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1aeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b01005c0 0x7fa1b0197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1aeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b01005c0 0x7fa1b0197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35130/0 (socket says 172.21.15.42:35130) 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1aeffd700 1 -- 172.21.15.42:0/1193926989 learned_addr learned my addr 172.21.15.42:0/1193926989 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1ae7fc700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1b0100f80 0x7fa1b01979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1af7fe700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b0101970 0x7fa1b019e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1aeffd700 1 -- 172.21.15.42:0/1193926989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1b0100f80 msgr2=0x7fa1b01979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1aeffd700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1b0100f80 0x7fa1b01979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1aeffd700 1 -- 172.21.15.42:0/1193926989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b0101970 msgr2=0x7fa1b019e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.583 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1aeffd700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b0101970 0x7fa1b019e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1aeffd700 1 -- 172.21.15.42:0/1193926989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a8009d00 con 0x7fa1b01005c0 2023-12-12T09:58:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1aeffd700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b01005c0 0x7fa1b0197480 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000d980 tx=0x7fa1a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa197fff700 1 -- 172.21.15.42:0/1193926989 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004220 con 0x7fa1b01005c0 2023-12-12T09:58:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa197fff700 1 -- 172.21.15.42:0/1193926989 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1a0007640 con 0x7fa1b01005c0 2023-12-12T09:58:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1b019f560 con 0x7fa1b01005c0 2023-12-12T09:58:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa197fff700 1 -- 172.21.15.42:0/1193926989 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0005690 con 0x7fa1b01005c0 2023-12-12T09:58:21.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.579+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1b019fa70 con 0x7fa1b01005c0 2023-12-12T09:58:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.580+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1b0068680 con 0x7fa1b01005c0 2023-12-12T09:58:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.582+0000 7fa197fff700 1 -- 172.21.15.42:0/1193926989 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fa1a0004750 con 0x7fa1b01005c0 2023-12-12T09:58:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.583+0000 7fa197fff700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1980a1510 0x7fa1980a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.583+0000 7fa1ae7fc700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1980a1510 0x7fa1980a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.583+0000 7fa197fff700 1 -- 172.21.15.42:0/1193926989 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa1a00cd440 con 0x7fa1b01005c0 2023-12-12T09:58:21.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.583+0000 7fa1ae7fc700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1980a1510 0x7fa1980a39d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009c60 tx=0x7fa1a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.588 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.586+0000 7fa197fff700 1 -- 172.21.15.42:0/1193926989 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa1a0096630 con 0x7fa1b01005c0 2023-12-12T09:58:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:21 smithi111 ceph-mon[102565]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:21 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3460762989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:58:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:21 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2176372157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:58:21.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:21 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/4056095368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:58:21.792 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.790+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1b010bc10 con 0x7fa1b01005c0 2023-12-12T09:58:21.794 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.792+0000 7fa197fff700 1 -- 172.21.15.42:0/1193926989 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+24476 (secure 0 0 0) 0x7fa1a0016090 con 0x7fa1b01005c0 2023-12-12T09:58:21.794 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T09:58:21.794 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":89,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T09:57:29.362715+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T09:58:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1980a1510 msgr2=0x7fa1980a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1980a1510 0x7fa1980a39d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4009c60 tx=0x7fa1a4009260 comp rx=0 tx=0).stop 2023-12-12T09:58:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b01005c0 msgr2=0x7fa1b0197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b01005c0 0x7fa1b0197480 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000d980 tx=0x7fa1a000dc90 comp rx=0 tx=0).stop 2023-12-12T09:58:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 shutdown_connections 2023-12-12T09:58:21.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fa1980a1510 0x7fa1980a39d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa1b0101970 0x7fa1b019e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa1b01005c0 0x7fa1b0197480 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 --2- 172.21.15.42:0/1193926989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa1b0100f80 0x7fa1b01979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 >> 172.21.15.42:0/1193926989 conn(0x7fa1b00fb0b0 msgr2=0x7fa1b00fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 shutdown_connections 2023-12-12T09:58:21.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.794+0000 7fa1b54a3700 1 -- 172.21.15.42:0/1193926989 wait complete. 2023-12-12T09:58:21.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:21 smithi042 ceph-mon[98894]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:21.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:21 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3460762989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:58:21.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:21 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2176372157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:58:21.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:21 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/4056095368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:58:21.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-12T09:56:37.902338+0000', 'modified': '2023-12-12T09:57:07.293702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24524, 'mds_1': 24508, 'mds_2': 24487, 'mds_3': 24482, 'mds_4': 24475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14616': {'gid': 14616, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.42:6839/229824980', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6838', 'nonce': 229824980}, {'type': 'v1', 'addr': '172.21.15.42:6839', 'nonce': 229824980}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14628': {'gid': 14628, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.42:6841/4221329195', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6840', 'nonce': 4221329195}, {'type': 'v1', 'addr': '172.21.15.42:6841', 'nonce': 4221329195}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24425': {'gid': 24425, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6833/1539682179', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6832', 'nonce': 1539682179}, {'type': 'v1', 'addr': '172.21.15.111:6833', 'nonce': 1539682179}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24430': {'gid': 24430, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.173:6833/2859393185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6832', 'nonce': 2859393185}, {'type': 'v1', 'addr': '172.21.15.173:6833', 'nonce': 2859393185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24452': {'gid': 24452, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.111:6835/4127941301', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6834', 'nonce': 4127941301}, {'type': 'v1', 'addr': '172.21.15.111:6835', 'nonce': 4127941301}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24475': {'gid': 24475, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.173:6835/3087189893', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 3087189893}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 3087189893}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'h', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.111:6837/3946741228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 3946741228}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 3946741228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.173:6837/1667746348', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6836', 'nonce': 1667746348}, {'type': 'v1', 'addr': '172.21.15.173:6837', 'nonce': 1667746348}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24508': {'gid': 24508, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.173:6839/2977372273', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6838', 'nonce': 2977372273}, {'type': 'v1', 'addr': '172.21.15.173:6839', 'nonce': 2977372273}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.111:6839/2430709127', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6838', 'nonce': 2430709127}, {'type': 'v1', 'addr': '172.21.15.111:6839', 'nonce': 2430709127}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-12T09:58:21.810 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2023-12-12T09:58:21.810 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2023-12-12T09:58:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:21 smithi173 ceph-mon[103308]: pgmap v315: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:21 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3460762989' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T09:58:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:21 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2176372157' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T09:58:21.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:21 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/4056095368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T09:58:21.937 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.934+0000 7f2ab5800700 1 -- 172.21.15.42:0/4118924587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab0101e80 msgr2=0x7f2ab0108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.937 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.934+0000 7f2ab5800700 1 --2- 172.21.15.42:0/4118924587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab0101e80 0x7f2ab0108730 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800b0d0 tx=0x7f2aa800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.937 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.934+0000 7f2ab5800700 1 -- 172.21.15.42:0/4118924587 shutdown_connections 2023-12-12T09:58:21.937 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.934+0000 7f2ab5800700 1 --2- 172.21.15.42:0/4118924587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab0101e80 0x7f2ab0108730 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.937 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.934+0000 7f2ab5800700 1 --2- 172.21.15.42:0/4118924587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab01002f0 0x7f2ab0100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.937 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.934+0000 7f2ab5800700 1 --2- 172.21.15.42:0/4118924587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ab01014e0 0x7f2ab0101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.937 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.934+0000 7f2ab5800700 1 -- 172.21.15.42:0/4118924587 >> 172.21.15.42:0/4118924587 conn(0x7f2ab00fb840 msgr2=0x7f2ab00fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.937 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.934+0000 7f2ab5800700 1 -- 172.21.15.42:0/4118924587 shutdown_connections 2023-12-12T09:58:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.934+0000 7f2ab5800700 1 -- 172.21.15.42:0/4118924587 wait complete. 2023-12-12T09:58:21.938 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.935+0000 7f2ab5800700 1 Processor -- start 2023-12-12T09:58:21.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.935+0000 7f2ab5800700 1 -- start start 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.935+0000 7f2ab5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab01002f0 0x7f2ab0069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.935+0000 7f2ab5800700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab01014e0 0x7f2ab019c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.935+0000 7f2ab5800700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ab0101e80 0x7f2ab019e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.935+0000 7f2ab5800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab019ef60 con 0x7f2ab01002f0 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.935+0000 7f2ab5800700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab019f0d0 con 0x7f2ab0101e80 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2ab5800700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab019f290 con 0x7f2ab01014e0 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab01002f0 0x7f2ab0069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab01002f0 0x7f2ab0069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35144/0 (socket says 172.21.15.42:35144) 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaeffd700 1 -- 172.21.15.42:0/1279211763 learned_addr learned my addr 172.21.15.42:0/1279211763 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab01014e0 0x7f2ab019c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ab0101e80 0x7f2ab019e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaeffd700 1 -- 172.21.15.42:0/1279211763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab01014e0 msgr2=0x7f2ab019c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab01014e0 0x7f2ab019c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaeffd700 1 -- 172.21.15.42:0/1279211763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ab0101e80 msgr2=0x7f2ab019e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ab0101e80 0x7f2ab019e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaeffd700 1 -- 172.21.15.42:0/1279211763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa8009d00 con 0x7f2ab01002f0 2023-12-12T09:58:21.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab01002f0 0x7f2ab0069130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000bf30 tx=0x7f2aa000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aa7fff700 1 -- 172.21.15.42:0/1279211763 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0004220 con 0x7f2ab01002f0 2023-12-12T09:58:21.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2ab5800700 1 -- 172.21.15.42:0/1279211763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab0196000 con 0x7f2ab01002f0 2023-12-12T09:58:21.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aa7fff700 1 -- 172.21.15.42:0/1279211763 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa0016e90 con 0x7f2ab01002f0 2023-12-12T09:58:21.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2aa7fff700 1 -- 172.21.15.42:0/1279211763 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00208b0 con 0x7f2ab01002f0 2023-12-12T09:58:21.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.936+0000 7f2ab5800700 1 -- 172.21.15.42:0/1279211763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab0196510 con 0x7f2ab01002f0 2023-12-12T09:58:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.939+0000 7f2aa7fff700 1 -- 172.21.15.42:0/1279211763 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f2aa0004380 con 0x7f2ab01002f0 2023-12-12T09:58:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.940+0000 7f2aa7fff700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a9c0a1510 0x7f2a9c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.940+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a9c0a1510 0x7f2a9c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.940+0000 7f2aa7fff700 1 -- 172.21.15.42:0/1279211763 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2aa00d4f90 con 0x7f2ab01002f0 2023-12-12T09:58:21.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.940+0000 7f2ac19f9b80 1 Processor -- start 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.940+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a9c0a1510 0x7f2a9c0a39d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00696f0 tx=0x7f2a98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.940+0000 7f2ac19f9b80 1 -- start start 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef964a10 0x5624ef964e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef965850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef9661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef8bfb00 con 0x5624ef965d90 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef8bd420 con 0x5624ef965370 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef8b8d10 con 0x5624ef964a10 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef9661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef9661f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35156/0 (socket says 172.21.15.42:35156) 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/2542718001 learned_addr learned my addr 172.21.15.42:0/2542718001 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:21.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef965850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef964a10 0x5624ef964e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/2542718001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef964a10 msgr2=0x5624ef964e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef964a10 0x5624ef964e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/2542718001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 msgr2=0x5624ef965850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef965850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/2542718001 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624ef7355e0 con 0x5624ef965d90 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef965850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef9661f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8015000 tx=0x7f2aa8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be86be56067fbe5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.941+0000 7f2aa57fa700 1 -- 172.21.15.42:0/2542718001 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800e040 con 0x5624ef965d90 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2aa57fa700 1 -- 172.21.15.42:0/2542718001 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa800baa0 con 0x5624ef965d90 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2aa57fa700 1 -- 172.21.15.42:0/2542718001 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8005070 con 0x5624ef965d90 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2542718001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 msgr2=0x5624ef9661f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.946 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef9661f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8015000 tx=0x7f2aa8005a20 comp rx=0 tx=0).stop 2023-12-12T09:58:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2542718001 shutdown_connections 2023-12-12T09:58:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef9661f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef964a10 0x5624ef964e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/2542718001 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef965850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2542718001 >> 172.21.15.42:0/2542718001 conn(0x5624ef893790 msgr2=0x5624ef95f380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2542718001 shutdown_connections 2023-12-12T09:58:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.942+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2542718001 wait complete. 2023-12-12T09:58:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2ac19f9b80 1 -- start start 2023-12-12T09:58:21.947 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2ac19f9b80 1 Processor -- start 2023-12-12T09:58:21.951 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef964a10 0x5624ef96e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.951 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef96e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.951 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965d90 0x5624ef975870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef872510 con 0x5624ef964a10 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef8721b0 con 0x5624ef965370 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef7378d0 con 0x5624ef965d90 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2ac19f9b80 10 client.0 _ll_register_callbacks cb 0x5624ef79da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aaeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef964a10 0x5624ef96e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aaeffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef964a10 0x5624ef96e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35158/0 (socket says 172.21.15.42:35158) 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aaeffd700 1 -- 172.21.15.42:0/1534076229 learned_addr learned my addr 172.21.15.42:0/1534076229 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965d90 0x5624ef975870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef96e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aaeffd700 1 -- 172.21.15.42:0/1534076229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965d90 msgr2=0x5624ef975870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:21.952 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965d90 0x5624ef975870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aaeffd700 1 -- 172.21.15.42:0/1534076229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 msgr2=0x5624ef96e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef96e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aaeffd700 1 -- 172.21.15.42:0/1534076229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab0109080 con 0x5624ef964a10 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.943+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef96e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef964a10 0x5624ef96e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0005e10 tx=0x7f2aa00a25f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daac8bd88173a6e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2a967fc700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2a967fc700 1 -- 172.21.15.42:0/1534076229 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa00d2de0 con 0x5624ef964a10 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2a967fc700 1 -- 172.21.15.42:0/1534076229 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2aa00a6040 con 0x5624ef964a10 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/1534076229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624ef957990 con 0x5624ef964a10 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2a967fc700 1 -- 172.21.15.42:0/1534076229 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aa00aa280 con 0x5624ef964a10 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2ac19f9b80 10 client.15420 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:21.953 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/1534076229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5624ef6e1d90 con 0x5624ef964a10 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2a967fc700 1 -- 172.21.15.42:0/1534076229 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5624ef957990 con 0x5624ef964a10 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2a967fc700 1 -- 172.21.15.42:0/1534076229 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2aa009dce0 con 0x5624ef964a10 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2ac19f9b80 10 client.15420 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2ac19f9b80 10 client.15420 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2ac19f9b80 20 client.15420 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x5624ef979e70 0x5624ef978980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2ac19f9b80 4 client.15420 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.944+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/1534076229 --> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624ef933ef0 con 0x5624ef979e70 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.945+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x5624ef979e70 0x5624ef978980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.945+0000 7f2a967fc700 10 client.15420 ms_handle_connect on v2:172.21.15.173:6834/3087189893 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.945+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x5624ef979e70 0x5624ef978980 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e5167820b802463 server_cookie=27646bf2c644f74d in_seq=0 out_seq=0 2023-12-12T09:58:21.954 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2a967fc700 1 -- 172.21.15.42:0/1534076229 <== mds.4 v2:172.21.15.173:6834/3087189893 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5624ef933ef0 con 0x5624ef979e70 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2a967fc700 10 client.15420 handle_command_reply: tid=0 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 client.15420 shutdown 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/1534076229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef964a10 msgr2=0x5624ef96e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef964a10 0x5624ef96e2f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0005e10 tx=0x7f2aa00a25f0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/1534076229 shutdown_connections 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x5624ef979e70 0x5624ef978980 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965d90 0x5624ef975870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef964a10 0x5624ef96e2f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/1534076229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef965370 0x5624ef96e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2a967fc700 0 client.15420 ms_handle_reset on v2:172.21.15.173:6834/3087189893 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/1534076229 >> 172.21.15.42:0/1534076229 conn(0x5624ef893790 msgr2=0x5624ef9637d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/1534076229 shutdown_connections 2023-12-12T09:58:21.955 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/1534076229 wait complete. 2023-12-12T09:58:21.956 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.946+0000 7f2ac19f9b80 20 client.15420 trim_cache size 0 max 16384 2023-12-12T09:58:21.972 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.964+0000 7f2ac19f9b80 1 Processor -- start 2023-12-12T09:58:21.972 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.964+0000 7f2ac19f9b80 1 -- start start 2023-12-12T09:58:21.972 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef966170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.972 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 0x5624ef975830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.972 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef96c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef8b9fd0 con 0x5624ef965d90 2023-12-12T09:58:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef8b4e30 con 0x5624ef964a10 2023-12-12T09:58:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef854360 con 0x5624ef965370 2023-12-12T09:58:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef96c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef96c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35170/0 (socket says 172.21.15.42:35170) 2023-12-12T09:58:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/2807630552 learned_addr learned my addr 172.21.15.42:0/2807630552 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/2807630552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef966170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.973 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/2807630552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 msgr2=0x5624ef975830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:21.974 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/2807630552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 0x5624ef975830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.974 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/2807630552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 msgr2=0x5624ef966170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:21.974 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/2807630552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef966170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.974 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/2807630552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624ef6cac10 con 0x5624ef965d90 2023-12-12T09:58:21.974 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/2807630552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef966170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:21.974 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/2807630552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef96c3d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009bf0 tx=0x7f2aa80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c0165cfefbccb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.974 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aa5ffb700 1 -- 172.21.15.42:0/2807630552 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800e040 con 0x5624ef965d90 2023-12-12T09:58:21.974 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aa5ffb700 1 -- 172.21.15.42:0/2807630552 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa8003d50 con 0x5624ef965d90 2023-12-12T09:58:21.974 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.965+0000 7f2aa5ffb700 1 -- 172.21.15.42:0/2807630552 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80256b0 con 0x5624ef965d90 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2807630552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 msgr2=0x5624ef96c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/2807630552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef96c3d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8009bf0 tx=0x7f2aa80099e0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2807630552 shutdown_connections 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/2807630552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef966170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/2807630552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef96c3d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/2807630552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 0x5624ef975830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2807630552 >> 172.21.15.42:0/2807630552 conn(0x5624ef893790 msgr2=0x5624ef96d870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2807630552 shutdown_connections 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/2807630552 wait complete. 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 -- start start 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.966+0000 7f2ac19f9b80 1 Processor -- start 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef974a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 0x5624ef974fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.975 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2ac19f9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef975650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef9cff30 con 0x5624ef965d90 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef9d0070 con 0x5624ef964a10 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2ac19f9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ef9d0230 con 0x5624ef965370 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2ac19f9b80 10 client.0 _ll_register_callbacks cb 0x5624ef79da30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef975650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaf7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef975650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35176/0 (socket says 172.21.15.42:35176) 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/3825345117 learned_addr learned my addr 172.21.15.42:0/3825345117 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef974a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 0x5624ef974fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/3825345117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 msgr2=0x5624ef974fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 0x5624ef974fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/3825345117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 msgr2=0x5624ef974a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef974a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaf7fe700 1 -- 172.21.15.42:0/3825345117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5624ef7355e0 con 0x5624ef965d90 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef974a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aae7fc700 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 0x5624ef974fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2aaf7fe700 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef975650 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8007b60 tx=0x7f2aa8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2220f082b9489d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2a967fc700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.967+0000 7f2a967fc700 1 -- 172.21.15.42:0/3825345117 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa800e040 con 0x5624ef965d90 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2a967fc700 1 -- 172.21.15.42:0/3825345117 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2aa8005250 con 0x5624ef965d90 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/3825345117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2aa800baa0 con 0x5624ef965d90 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2a967fc700 1 -- 172.21.15.42:0/3825345117 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aa8005410 con 0x5624ef965d90 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2ac19f9b80 10 client.15426 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/3825345117 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5624ef708200 con 0x5624ef965d90 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2a967fc700 1 -- 172.21.15.42:0/3825345117 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2aa800baa0 con 0x5624ef965d90 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2a967fc700 1 -- 172.21.15.42:0/3825345117 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f2aa800b100 con 0x5624ef965d90 2023-12-12T09:58:21.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2ac19f9b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2ac19f9b80 10 client.15426 resolve_mds: resolved name 'f' to daemon mds.f 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2ac19f9b80 20 client.15426 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x5624ef9d5100 0x5624ef9d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2ac19f9b80 4 client.15426 mds_command: new command op to 24475 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/3825345117 --> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5624ef87ef80 con 0x5624ef9d5100 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.968+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x5624ef9d5100 0x5624ef9d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2aaeffd700 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x5624ef9d5100 0x5624ef9d15a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2b446fc48733385 server_cookie=757b731f2199e4b6 in_seq=0 out_seq=0 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2a967fc700 10 client.15426 ms_handle_connect on v2:172.21.15.173:6834/3087189893 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2a967fc700 1 -- 172.21.15.42:0/3825345117 <== mds.4 v2:172.21.15.173:6834/3087189893 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5624ef87ef80 con 0x5624ef9d5100 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2a967fc700 10 client.15426 handle_command_reply: tid=0 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 client.15426 shutdown 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/3825345117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 msgr2=0x5624ef975650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef975650 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8007b60 tx=0x7f2aa8003f40 comp rx=0 tx=0).stop 2023-12-12T09:58:21.978 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/3825345117 shutdown_connections 2023-12-12T09:58:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5624ef964a10 0x5624ef974a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.173:6834/3087189893,v1:172.21.15.173:6835/3087189893] conn(0x5624ef9d5100 0x5624ef9d15a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5624ef965d90 0x5624ef975650 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 --2- 172.21.15.42:0/3825345117 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5624ef965370 0x5624ef974fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2a967fc700 0 client.15426 ms_handle_reset on v2:172.21.15.173:6834/3087189893 2023-12-12T09:58:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/3825345117 >> 172.21.15.42:0/3825345117 conn(0x5624ef893790 msgr2=0x5624ef95ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/3825345117 shutdown_connections 2023-12-12T09:58:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 1 -- 172.21.15.42:0/3825345117 wait complete. 2023-12-12T09:58:21.979 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.969+0000 7f2ac19f9b80 20 client.15426 trim_cache size 0 max 16384 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 4, 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24475, 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 75.679440485000001, 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 123.432804463 2023-12-12T09:58:21.981 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:58:21.982 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 -- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a9c0a1510 msgr2=0x7f2a9c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.982 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a9c0a1510 0x7f2a9c0a39d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00696f0 tx=0x7f2a98009260 comp rx=0 tx=0).stop 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 -- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab01002f0 msgr2=0x7f2ab0069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab01002f0 0x7f2ab0069130 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000bf30 tx=0x7f2aa000dbb0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 -- 172.21.15.42:0/1279211763 shutdown_connections 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f2a9c0a1510 0x7f2a9c0a39d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ab0101e80 0x7f2ab019e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ab01002f0 0x7f2ab0069130 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 --2- 172.21.15.42:0/1279211763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab01014e0 0x7f2ab019c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 -- 172.21.15.42:0/1279211763 >> 172.21.15.42:0/1279211763 conn(0x7f2ab00fb840 msgr2=0x7f2ab0104860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 -- 172.21.15.42:0/1279211763 shutdown_connections 2023-12-12T09:58:21.983 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:21.970+0000 7f2ab5800700 1 -- 172.21.15.42:0/1279211763 wait complete. 2023-12-12T09:58:21.984 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-12-12T09:58:22.114 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 -- 172.21.15.42:0/2046570571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc104930 msgr2=0x7fe7bc106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 --2- 172.21.15.42:0/2046570571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc104930 0x7fe7bc106d20 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009990 tx=0x7fe7b0009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 -- 172.21.15.42:0/2046570571 shutdown_connections 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 --2- 172.21.15.42:0/2046570571 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc102000 0x7fe7bc1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 --2- 172.21.15.42:0/2046570571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7bc107260 0x7fe7bc1096d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 --2- 172.21.15.42:0/2046570571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc104930 0x7fe7bc106d20 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 -- 172.21.15.42:0/2046570571 >> 172.21.15.42:0/2046570571 conn(0x7fe7bc0fb9f0 msgr2=0x7fe7bc0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 -- 172.21.15.42:0/2046570571 shutdown_connections 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 -- 172.21.15.42:0/2046570571 wait complete. 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 Processor -- start 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.110+0000 7fe7c2365700 1 -- start start 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7c2365700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc102000 0x7fe7bc197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7c2365700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc104930 0x7fe7bc197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.115 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7c2365700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7bc107260 0x7fe7bc19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7c2365700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc19ef70 con 0x7fe7bc102000 2023-12-12T09:58:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7c2365700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc19f0b0 con 0x7fe7bc104930 2023-12-12T09:58:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7c2365700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc19f240 con 0x7fe7bc107260 2023-12-12T09:58:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc102000 0x7fe7bc197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc102000 0x7fe7bc197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35184/0 (socket says 172.21.15.42:35184) 2023-12-12T09:58:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bbfff700 1 -- 172.21.15.42:0/3612956585 learned_addr learned my addr 172.21.15.42:0/3612956585 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc104930 0x7fe7bc197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.116 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7c0902700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7bc107260 0x7fe7bc19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bbfff700 1 -- 172.21.15.42:0/3612956585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7bc107260 msgr2=0x7fe7bc19e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7bc107260 0x7fe7bc19e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bbfff700 1 -- 172.21.15.42:0/3612956585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc104930 msgr2=0x7fe7bc197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc104930 0x7fe7bc197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bbfff700 1 -- 172.21.15.42:0/3612956585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b00095f0 con 0x7fe7bc102000 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7c0902700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7bc107260 0x7fe7bc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc102000 0x7fe7bc197440 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c9b0 tx=0x7fe7ac00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.111+0000 7fe7b97fa700 1 -- 172.21.15.42:0/3612956585 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac007b00 con 0x7fe7bc102000 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.112+0000 7fe7b97fa700 1 -- 172.21.15.42:0/3612956585 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7ac007c60 con 0x7fe7bc102000 2023-12-12T09:58:22.117 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.112+0000 7fe7c2365700 1 -- 172.21.15.42:0/3612956585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc19f520 con 0x7fe7bc102000 2023-12-12T09:58:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.112+0000 7fe7b97fa700 1 -- 172.21.15.42:0/3612956585 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac018660 con 0x7fe7bc102000 2023-12-12T09:58:22.118 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.112+0000 7fe7c2365700 1 -- 172.21.15.42:0/3612956585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc19fa30 con 0x7fe7bc102000 2023-12-12T09:58:22.125 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.114+0000 7fe7b97fa700 1 -- 172.21.15.42:0/3612956585 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fe7ac00f450 con 0x7fe7bc102000 2023-12-12T09:58:22.125 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.114+0000 7fe7b97fa700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe7a40a1510 0x7fe7a40a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.125 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7b97fa700 1 -- 172.21.15.42:0/3612956585 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe7ac0cd150 con 0x7fe7bc102000 2023-12-12T09:58:22.125 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe7a40a1510 0x7fe7a40a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.125 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe7a40a1510 0x7fe7a40a39d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000b5c0 tx=0x7fe7b0003790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7ce55eb80 1 Processor -- start 2023-12-12T09:58:22.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7ce55eb80 1 -- start start 2023-12-12T09:58:22.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 0x55f4e5ace7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acece0 0x55f4e5acf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 0x55f4e5ad0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e5a29b00 con 0x55f4e5acece0 2023-12-12T09:58:22.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e5a27420 con 0x55f4e5acfbd0 2023-12-12T09:58:22.126 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e5a22d10 con 0x55f4e5ace380 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.115+0000 7fe7bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acece0 0x55f4e5acf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7bb7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acece0 0x55f4e5acf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35194/0 (socket says 172.21.15.42:35194) 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7bb7fe700 1 -- 172.21.15.42:0/2939279293 learned_addr learned my addr 172.21.15.42:0/2939279293 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 0x55f4e5ace7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7c0902700 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 0x55f4e5ad0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7bb7fe700 1 -- 172.21.15.42:0/2939279293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 msgr2=0x55f4e5ace7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 0x55f4e5ace7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7bb7fe700 1 -- 172.21.15.42:0/2939279293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 msgr2=0x55f4e5ad0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 0x55f4e5ad0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7bb7fe700 1 -- 172.21.15.42:0/2939279293 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4e589f5e0 con 0x55f4e5acece0 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7c0902700 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 0x55f4e5ad0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acece0 0x55f4e5acf5f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0015000 tx=0x7fe7b0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f666153d77383d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.127 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7a27fc700 1 -- 172.21.15.42:0/2939279293 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b002a070 con 0x55f4e5acece0 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7a27fc700 1 -- 172.21.15.42:0/2939279293 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7b001e040 con 0x55f4e5acece0 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.116+0000 7fe7a27fc700 1 -- 172.21.15.42:0/2939279293 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0023a80 con 0x55f4e5acece0 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2939279293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acece0 msgr2=0x55f4e5acf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acece0 0x55f4e5acf5f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0015000 tx=0x7fe7b0020000 comp rx=0 tx=0).stop 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2939279293 shutdown_connections 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 0x55f4e5ad0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 0x55f4e5ace7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/2939279293 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acece0 0x55f4e5acf5f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2939279293 >> 172.21.15.42:0/2939279293 conn(0x55f4e59fd790 msgr2=0x55f4e5acad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2939279293 shutdown_connections 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2939279293 wait complete. 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.117+0000 7fe7ce55eb80 1 -- start start 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7ce55eb80 1 Processor -- start 2023-12-12T09:58:22.128 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 0x55f4e5ad85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e59dc510 con 0x55f4e5ace380 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e59dc1b0 con 0x55f4e5acece0 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e58a18d0 con 0x55f4e5acfbd0 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7ce55eb80 10 client.0 _ll_register_callbacks cb 0x55f4e5907a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35198/0 (socket says 172.21.15.42:35198) 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bbfff700 1 -- 172.21.15.42:0/1925854597 learned_addr learned my addr 172.21.15.42:0/1925854597 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 0x55f4e5ad85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7c0902700 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bbfff700 1 -- 172.21.15.42:0/1925854597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acfbd0 msgr2=0x55f4e5adf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bbfff700 1 -- 172.21.15.42:0/1925854597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 msgr2=0x55f4e5ad85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.129 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 0x55f4e5ad85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.130 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bbfff700 1 -- 172.21.15.42:0/1925854597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc10a2c0 con 0x55f4e5ace380 2023-12-12T09:58:22.130 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad8090 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00afd0 tx=0x7fe7ac09a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dd9ee85264b2f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.130 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.118+0000 7fe79b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:22.130 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1925854597 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac09e150 con 0x55f4e5ace380 2023-12-12T09:58:22.130 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1925854597 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe7ac0a2390 con 0x55f4e5ace380 2023-12-12T09:58:22.130 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1925854597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4e5ac1af0 con 0x55f4e5ace380 2023-12-12T09:58:22.130 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1925854597 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7ac09a260 con 0x55f4e5ace380 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7ce55eb80 10 client.15438 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1925854597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f4e584bd90 con 0x55f4e5ace380 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1925854597 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7ac063080 con 0x55f4e5ace380 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1925854597 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe7ac0a1f00 con 0x55f4e5ace380 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7ce55eb80 10 client.15438 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7ce55eb80 10 client.15438 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7ce55eb80 20 client.15438 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55f4e5ae3b80 0x55f4e5ae26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7ce55eb80 4 client.15438 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:58:22.131 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1925854597 --> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4e5a9def0 con 0x55f4e5ae3b80 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.119+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55f4e5ae3b80 0x55f4e5ae26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.120+0000 7fe79b7fe700 10 client.15438 ms_handle_connect on v2:172.21.15.111:6836/3946741228 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.120+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55f4e5ae3b80 0x55f4e5ae26c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b8ae702025cfd586 server_cookie=754b584cf5f8add1 in_seq=0 out_seq=0 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1925854597 <== mds.3 v2:172.21.15.111:6836/3946741228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4e5a9def0 con 0x55f4e5ae3b80 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe79b7fe700 10 client.15438 handle_command_reply: tid=0 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 client.15438 shutdown 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1925854597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 msgr2=0x55f4e5ad8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad8090 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00afd0 tx=0x7fe7ac09a800 comp rx=0 tx=0).stop 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1925854597 shutdown_connections 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55f4e5ae3b80 0x55f4e5ae26c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.132 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad8090 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.133 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe79b7fe700 0 client.15438 ms_handle_reset on v2:172.21.15.111:6836/3946741228 2023-12-12T09:58:22.133 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1925854597 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 0x55f4e5ad85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.133 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1925854597 >> 172.21.15.42:0/1925854597 conn(0x55f4e59fd790 msgr2=0x55f4e5ac99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.133 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1925854597 shutdown_connections 2023-12-12T09:58:22.133 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1925854597 wait complete. 2023-12-12T09:58:22.133 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.121+0000 7fe7ce55eb80 20 client.15438 trim_cache size 0 max 16384 2023-12-12T09:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7ce55eb80 1 Processor -- start 2023-12-12T09:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7ce55eb80 1 -- start start 2023-12-12T09:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acece0 0x55f4e5ad3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e5a23fa0 con 0x55f4e5ace380 2023-12-12T09:58:22.147 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e5a1ee00 con 0x55f4e5acfbd0 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e59c7b50 con 0x55f4e5acece0 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bbfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35206/0 (socket says 172.21.15.42:35206) 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bbfff700 1 -- 172.21.15.42:0/2300935379 learned_addr learned my addr 172.21.15.42:0/2300935379 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/2300935379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acece0 0x55f4e5ad3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7c0902700 1 --2- 172.21.15.42:0/2300935379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bbfff700 1 -- 172.21.15.42:0/2300935379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acece0 msgr2=0x55f4e5ad3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/2300935379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acece0 0x55f4e5ad3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bbfff700 1 -- 172.21.15.42:0/2300935379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 msgr2=0x55f4e5adf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/2300935379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bbfff700 1 -- 172.21.15.42:0/2300935379 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4e5834c10 con 0x55f4e5ace380 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.139+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/2300935379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad2b40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac099f90 tx=0x7fe7ac09a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a504447a6248efb8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7a2ffd700 1 -- 172.21.15.42:0/2300935379 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0994f0 con 0x55f4e5ace380 2023-12-12T09:58:22.148 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7a2ffd700 1 -- 172.21.15.42:0/2300935379 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f4e5ac1af0 con 0x55f4e5ace380 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7a2ffd700 1 -- 172.21.15.42:0/2300935379 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0a1f00 con 0x55f4e5ace380 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2300935379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 msgr2=0x55f4e5ad2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/2300935379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad2b40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac099f90 tx=0x7fe7ac09a180 comp rx=0 tx=0).stop 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2300935379 shutdown_connections 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/2300935379 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/2300935379 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5ace380 0x55f4e5ad2b40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/2300935379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5acece0 0x55f4e5ad3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2300935379 >> 172.21.15.42:0/2300935379 conn(0x55f4e59fd790 msgr2=0x55f4e5ad6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2300935379 shutdown_connections 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.140+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/2300935379 wait complete. 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7ce55eb80 1 -- start start 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7ce55eb80 1 Processor -- start 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 0x55f4e5adf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 0x55f4e5adbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.149 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7ce55eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e5adc850 con 0x55f4e5acfbd0 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e5adc9c0 con 0x55f4e5acece0 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7ce55eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4e5b39c40 con 0x55f4e5ace380 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7ce55eb80 10 client.0 _ll_register_callbacks cb 0x55f4e5907a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7c0902700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7c0902700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35214/0 (socket says 172.21.15.42:35214) 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7c0902700 1 -- 172.21.15.42:0/1907041179 learned_addr learned my addr 172.21.15.42:0/1907041179 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7bb7fe700 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 0x55f4e5adbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.141+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 0x55f4e5adf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe7c0902700 1 -- 172.21.15.42:0/1907041179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 msgr2=0x55f4e5adf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe7c0902700 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 0x55f4e5adf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe7c0902700 1 -- 172.21.15.42:0/1907041179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 msgr2=0x55f4e5adbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.150 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe7c0902700 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 0x55f4e5adbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe7c0902700 1 -- 172.21.15.42:0/1907041179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4e584bd90 con 0x55f4e5acfbd0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe7c0902700 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adc310 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40078f0 tx=0x7fe7b4007c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a250550606d0d6c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe79b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1907041179 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400e040 con 0x55f4e5acfbd0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1907041179 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe7b4004db0 con 0x55f4e5acfbd0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1907041179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7b001e040 con 0x55f4e5acfbd0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1907041179 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7b40136a0 con 0x55f4e5acfbd0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe7ce55eb80 10 client.15444 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1907041179 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f4e5872200 con 0x55f4e5acfbd0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.142+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1907041179 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7b001e040 con 0x55f4e5acfbd0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1907041179 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fe7b4017a00 con 0x55f4e5acfbd0 2023-12-12T09:58:22.151 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe7ce55eb80 10 client.15444 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe7ce55eb80 10 client.15444 resolve_mds: resolved name 'h' to daemon mds.h 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe7ce55eb80 20 client.15444 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55f4e5b3ee80 0x55f4e5b3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe7ce55eb80 4 client.15444 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1907041179 --> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f4e59e8f80 con 0x55f4e5b3ee80 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55f4e5b3ee80 0x55f4e5b3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe79b7fe700 10 client.15444 ms_handle_connect on v2:172.21.15.111:6836/3946741228 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe7bbfff700 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55f4e5b3ee80 0x55f4e5b3b320 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3164b51b4fca226a server_cookie=55a897b988612695 in_seq=0 out_seq=0 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe79b7fe700 1 -- 172.21.15.42:0/1907041179 <== mds.3 v2:172.21.15.111:6836/3946741228 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x55f4e59e8f80 con 0x55f4e5b3ee80 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe79b7fe700 10 client.15444 handle_command_reply: tid=0 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.143+0000 7fe7ce55eb80 1 client.15444 shutdown 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1907041179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acfbd0 msgr2=0x55f4e5adc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adc310 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40078f0 tx=0x7fe7b4007c00 comp rx=0 tx=0).stop 2023-12-12T09:58:22.152 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1907041179 shutdown_connections 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.111:6836/3946741228,v1:172.21.15.111:6837/3946741228] conn(0x55f4e5b3ee80 0x55f4e5b3b320 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f4e5acfbd0 0x55f4e5adc310 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe79b7fe700 0 client.15444 ms_handle_reset on v2:172.21.15.111:6836/3946741228 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f4e5ace380 0x55f4e5adf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 --2- 172.21.15.42:0/1907041179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f4e5acece0 0x55f4e5adbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1907041179 >> 172.21.15.42:0/1907041179 conn(0x55f4e59fd790 msgr2=0x55f4e5ad2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1907041179 shutdown_connections 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 1 -- 172.21.15.42:0/1907041179 wait complete. 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7ce55eb80 20 client.15444 trim_cache size 0 max 16384 2023-12-12T09:58:22.153 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 3, 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24482, 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 76.857082667, 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 119.20648779 2023-12-12T09:58:22.154 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:58:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7c2365700 1 -- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe7a40a1510 msgr2=0x7fe7a40a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7c2365700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe7a40a1510 0x7fe7a40a39d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000b5c0 tx=0x7fe7b0003790 comp rx=0 tx=0).stop 2023-12-12T09:58:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7c2365700 1 -- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc102000 msgr2=0x7fe7bc197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.155 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.144+0000 7fe7c2365700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc102000 0x7fe7bc197440 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c9b0 tx=0x7fe7ac00ccc0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.145+0000 7fe7c2365700 1 -- 172.21.15.42:0/3612956585 shutdown_connections 2023-12-12T09:58:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.145+0000 7fe7c2365700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fe7a40a1510 0x7fe7a40a39d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.145+0000 7fe7c2365700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7bc107260 0x7fe7bc19e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.145+0000 7fe7c2365700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7bc102000 0x7fe7bc197440 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.145+0000 7fe7c2365700 1 --2- 172.21.15.42:0/3612956585 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7bc104930 0x7fe7bc197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.145+0000 7fe7c2365700 1 -- 172.21.15.42:0/3612956585 >> 172.21.15.42:0/3612956585 conn(0x7fe7bc0fb9f0 msgr2=0x7fe7bc102900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.145+0000 7fe7c2365700 1 -- 172.21.15.42:0/3612956585 shutdown_connections 2023-12-12T09:58:22.156 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.145+0000 7fe7c2365700 1 -- 172.21.15.42:0/3612956585 wait complete. 2023-12-12T09:58:22.160 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.286+0000 7f9a17817700 1 -- 172.21.15.42:0/1084148903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 msgr2=0x7f9a10101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.286+0000 7f9a17817700 1 --2- 172.21.15.42:0/1084148903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 0x7f9a10101ad0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9a000099c0 tx=0x7f9a00009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.286+0000 7f9a17817700 1 -- 172.21.15.42:0/1084148903 shutdown_connections 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.286+0000 7f9a17817700 1 --2- 172.21.15.42:0/1084148903 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a10100480 0x7f9a101008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.286+0000 7f9a17817700 1 --2- 172.21.15.42:0/1084148903 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a10102010 0x7f9a101088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.286+0000 7f9a17817700 1 --2- 172.21.15.42:0/1084148903 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 0x7f9a10101ad0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.286+0000 7f9a17817700 1 -- 172.21.15.42:0/1084148903 >> 172.21.15.42:0/1084148903 conn(0x7f9a100fb9f0 msgr2=0x7f9a100fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 -- 172.21.15.42:0/1084148903 shutdown_connections 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 -- 172.21.15.42:0/1084148903 wait complete. 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 Processor -- start 2023-12-12T09:58:22.291 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 -- start start 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a10100480 0x7f9a10199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 0x7f9a10199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a10102010 0x7f9a101a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a101a11b0 con 0x7f9a10101670 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a101a12f0 con 0x7f9a10100480 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.287+0000 7f9a17817700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a101a1480 con 0x7f9a10102010 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a14db2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 0x7f9a10199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a14db2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 0x7f9a10199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35226/0 (socket says 172.21.15.42:35226) 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a14db2700 1 -- 172.21.15.42:0/2964988194 learned_addr learned my addr 172.21.15.42:0/2964988194 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a155b3700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a10100480 0x7f9a10199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a15db4700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a10102010 0x7f9a101a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a14db2700 1 -- 172.21.15.42:0/2964988194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a10102010 msgr2=0x7f9a101a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a14db2700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a10102010 0x7f9a101a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a14db2700 1 -- 172.21.15.42:0/2964988194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a10100480 msgr2=0x7f9a10199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a14db2700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a10100480 0x7f9a10199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a14db2700 1 -- 172.21.15.42:0/2964988194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000095f0 con 0x7f9a10101670 2023-12-12T09:58:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a14db2700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 0x7f9a10199bc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009fd0 tx=0x7f9a00005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a067fc700 1 -- 172.21.15.42:0/2964988194 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0001d070 con 0x7f9a10101670 2023-12-12T09:58:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a17817700 1 -- 172.21.15.42:0/2964988194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a101a1700 con 0x7f9a10101670 2023-12-12T09:58:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a067fc700 1 -- 172.21.15.42:0/2964988194 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a000059a0 con 0x7f9a10101670 2023-12-12T09:58:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a067fc700 1 -- 172.21.15.42:0/2964988194 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a000173f0 con 0x7f9a10101670 2023-12-12T09:58:22.293 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.288+0000 7f9a17817700 1 -- 172.21.15.42:0/2964988194 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a101a1bb0 con 0x7f9a10101670 2023-12-12T09:58:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.290+0000 7f9a067fc700 1 -- 172.21.15.42:0/2964988194 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f9a00022670 con 0x7f9a10101670 2023-12-12T09:58:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.291+0000 7f9a067fc700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99fc0a1510 0x7f99fc0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.291+0000 7f9a067fc700 1 -- 172.21.15.42:0/2964988194 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a000ce230 con 0x7f9a10101670 2023-12-12T09:58:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.291+0000 7f9a155b3700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99fc0a1510 0x7f99fc0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a155b3700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99fc0a1510 0x7f99fc0a39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9a101014d0 tx=0x7f9a0c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a23a10b80 1 Processor -- start 2023-12-12T09:58:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a23a10b80 1 -- start start 2023-12-12T09:58:22.299 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577380 0x55abc55777a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577ce0 0x55abc55785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 0x55abc5579050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc54d2b00 con 0x55abc5577ce0 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc54d0420 con 0x55abc5578bd0 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc54cbd10 con 0x55abc5577380 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a14db2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577ce0 0x55abc55785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a14db2700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577ce0 0x55abc55785f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35234/0 (socket says 172.21.15.42:35234) 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.292+0000 7f9a14db2700 1 -- 172.21.15.42:0/3419603286 learned_addr learned my addr 172.21.15.42:0/3419603286 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f9a15db4700 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 0x55abc5579050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f9a155b3700 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577380 0x55abc55777a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f9a14db2700 1 -- 172.21.15.42:0/3419603286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577380 msgr2=0x55abc55777a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f9a14db2700 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577380 0x55abc55777a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f9a14db2700 1 -- 172.21.15.42:0/3419603286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 msgr2=0x55abc5579050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.300 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f9a14db2700 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 0x55abc5579050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f9a14db2700 1 -- 172.21.15.42:0/3419603286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc53485e0 con 0x55abc5577ce0 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f9a15db4700 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 0x55abc5579050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f9a14db2700 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577ce0 0x55abc55785f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9a000a2fd0 tx=0x7f9a0009b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff00056ab8e1791 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f99fb7fe700 1 -- 172.21.15.42:0/3419603286 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0009f170 con 0x55abc5577ce0 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f99fb7fe700 1 -- 172.21.15.42:0/3419603286 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a0009b900 con 0x55abc5577ce0 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.293+0000 7f99fb7fe700 1 -- 172.21.15.42:0/3419603286 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0009a6f0 con 0x55abc5577ce0 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3419603286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577ce0 msgr2=0x55abc55785f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577ce0 0x55abc55785f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9a000a2fd0 tx=0x7f9a0009b100 comp rx=0 tx=0).stop 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3419603286 shutdown_connections 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 0x55abc5579050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577380 0x55abc55777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3419603286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577ce0 0x55abc55785f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3419603286 >> 172.21.15.42:0/3419603286 conn(0x55abc54a6790 msgr2=0x55abc5573d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3419603286 shutdown_connections 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3419603286 wait complete. 2023-12-12T09:58:22.301 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 -- start start 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.294+0000 7f9a23a10b80 1 Processor -- start 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5580de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 0x55abc5581320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 0x55abc5588360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc5485510 con 0x55abc5577380 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc54851b0 con 0x55abc5577ce0 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc534a8d0 con 0x55abc5578bd0 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a23a10b80 10 client.0 _ll_register_callbacks cb 0x55abc53b0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a155b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5580de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a155b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5580de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35238/0 (socket says 172.21.15.42:35238) 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a155b3700 1 -- 172.21.15.42:0/2450646360 learned_addr learned my addr 172.21.15.42:0/2450646360 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a15db4700 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 0x55abc5588360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a14db2700 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 0x55abc5581320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a155b3700 1 -- 172.21.15.42:0/2450646360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 msgr2=0x55abc5588360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a155b3700 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 0x55abc5588360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a155b3700 1 -- 172.21.15.42:0/2450646360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 msgr2=0x55abc5581320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a155b3700 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 0x55abc5581320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a155b3700 1 -- 172.21.15.42:0/2450646360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a101a1bb0 con 0x55abc5577380 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a155b3700 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5580de0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c01d9e0 tx=0x7f9a0c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e5dedc5d8113b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f99f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f99f8ff9700 1 -- 172.21.15.42:0/2450646360 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c01dea0 con 0x55abc5577380 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f9a23a10b80 1 -- 172.21.15.42:0/2450646360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abc556aaf0 con 0x55abc5577380 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.295+0000 7f99f8ff9700 1 -- 172.21.15.42:0/2450646360 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a0c010840 con 0x55abc5577380 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f99f8ff9700 1 -- 172.21.15.42:0/2450646360 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a0c010a00 con 0x55abc5577380 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f9a23a10b80 10 client.15456 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:22.303 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f9a23a10b80 1 -- 172.21.15.42:0/2450646360 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55abc52f4d90 con 0x55abc5577380 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f99f8ff9700 1 -- 172.21.15.42:0/2450646360 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55abc556aaf0 con 0x55abc5577380 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f99f8ff9700 1 -- 172.21.15.42:0/2450646360 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9a0c01bcd0 con 0x55abc5577380 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f9a23a10b80 10 client.15456 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f9a23a10b80 10 client.15456 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f9a23a10b80 20 client.15456 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55abc558ca90 0x55abc558b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f9a23a10b80 4 client.15456 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f9a23a10b80 1 -- 172.21.15.42:0/2450646360 --> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abc5546ef0 con 0x55abc558ca90 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.296+0000 7f9a14db2700 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55abc558ca90 0x55abc558b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.297+0000 7f99f8ff9700 10 client.15456 ms_handle_connect on v2:172.21.15.173:6836/1667746348 2023-12-12T09:58:22.304 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.297+0000 7f9a14db2700 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55abc558ca90 0x55abc558b560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a648e2c35d35218 server_cookie=fd7f672ff8a53a09 in_seq=0 out_seq=0 2023-12-12T09:58:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.297+0000 7f99f8ff9700 1 -- 172.21.15.42:0/2450646360 <== mds.2 v2:172.21.15.173:6836/1667746348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55abc5546ef0 con 0x55abc558ca90 2023-12-12T09:58:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.297+0000 7f99f8ff9700 10 client.15456 handle_command_reply: tid=0 2023-12-12T09:58:22.305 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.297+0000 7f9a23a10b80 1 client.15456 shutdown 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.297+0000 7f9a23a10b80 1 -- 172.21.15.42:0/2450646360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 msgr2=0x55abc5580de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.297+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5580de0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c01d9e0 tx=0x7f9a0c0047c0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f9a23a10b80 1 -- 172.21.15.42:0/2450646360 shutdown_connections 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55abc558ca90 0x55abc558b560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 0x55abc5588360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f99f8ff9700 0 client.15456 ms_handle_reset on v2:172.21.15.173:6836/1667746348 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5580de0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/2450646360 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 0x55abc5581320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f9a23a10b80 1 -- 172.21.15.42:0/2450646360 >> 172.21.15.42:0/2450646360 conn(0x55abc54a6790 msgr2=0x55abc55729f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f9a23a10b80 1 -- 172.21.15.42:0/2450646360 shutdown_connections 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f9a23a10b80 1 -- 172.21.15.42:0/2450646360 wait complete. 2023-12-12T09:58:22.306 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.298+0000 7f9a23a10b80 20 client.15456 trim_cache size 0 max 16384 2023-12-12T09:58:22.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.316+0000 7f9a23a10b80 1 Processor -- start 2023-12-12T09:58:22.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.316+0000 7f9a23a10b80 1 -- start start 2023-12-12T09:58:22.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.316+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5581a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.316+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 0x55abc5588320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.316+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 0x55abc557ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.316+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc5467360 con 0x55abc5577380 2023-12-12T09:58:22.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.316+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc546e830 con 0x55abc5577ce0 2023-12-12T09:58:22.325 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.316+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc558a0c0 con 0x55abc5578bd0 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a155b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5581a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a155b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5581a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35246/0 (socket says 172.21.15.42:35246) 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a155b3700 1 -- 172.21.15.42:0/1885130237 learned_addr learned my addr 172.21.15.42:0/1885130237 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a14db2700 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 0x55abc5588320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a15db4700 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 0x55abc557ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a155b3700 1 -- 172.21.15.42:0/1885130237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 msgr2=0x55abc557ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a155b3700 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 0x55abc557ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a155b3700 1 -- 172.21.15.42:0/1885130237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 msgr2=0x55abc5588320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a155b3700 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 0x55abc5588320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a155b3700 1 -- 172.21.15.42:0/1885130237 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abc52ddc10 con 0x55abc5577380 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a14db2700 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 0x55abc5588320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f9a155b3700 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5581a40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00a9e0 tx=0x7f9a0c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3663f330de91922 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.326 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f99fbfff700 1 -- 172.21.15.42:0/1885130237 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c0259e0 con 0x55abc5577380 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f99fbfff700 1 -- 172.21.15.42:0/1885130237 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a0c014070 con 0x55abc5577380 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.317+0000 7f99fbfff700 1 -- 172.21.15.42:0/1885130237 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c035690 con 0x55abc5577380 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 -- 172.21.15.42:0/1885130237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 msgr2=0x55abc5581a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5581a40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00a9e0 tx=0x7f9a0c0046d0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 -- 172.21.15.42:0/1885130237 shutdown_connections 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5578bd0 0x55abc557ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5581a40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/1885130237 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5577ce0 0x55abc5588320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 -- 172.21.15.42:0/1885130237 >> 172.21.15.42:0/1885130237 conn(0x55abc54a6790 msgr2=0x55abc557f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 -- 172.21.15.42:0/1885130237 shutdown_connections 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 -- 172.21.15.42:0/1885130237 wait complete. 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 -- start start 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.318+0000 7f9a23a10b80 1 Processor -- start 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5587b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577ce0 0x55abc55880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.327 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a23a10b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 0x55abc55846c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc5584c00 con 0x55abc5577380 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc5584d70 con 0x55abc5578bd0 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a23a10b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abc5584ee0 con 0x55abc5577ce0 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a23a10b80 10 client.0 _ll_register_callbacks cb 0x55abc53b0a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a155b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5587b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a155b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5587b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35252/0 (socket says 172.21.15.42:35252) 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a155b3700 1 -- 172.21.15.42:0/3883335558 learned_addr learned my addr 172.21.15.42:0/3883335558 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a14db2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577ce0 0x55abc55880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a15db4700 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 0x55abc55846c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a155b3700 1 -- 172.21.15.42:0/3883335558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577ce0 msgr2=0x55abc55880c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a155b3700 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577ce0 0x55abc55880c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a155b3700 1 -- 172.21.15.42:0/3883335558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 msgr2=0x55abc55846c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.328 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a155b3700 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 0x55abc55846c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a155b3700 1 -- 172.21.15.42:0/3883335558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55abc52f4d90 con 0x55abc5577380 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f9a155b3700 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5587b80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000c00 tx=0x7f9a0c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da95cfb0d8990cde server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f99f8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.319+0000 7f99f8ff9700 1 -- 172.21.15.42:0/3883335558 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c025be0 con 0x55abc5577380 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3883335558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a0009b900 con 0x55abc5577380 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f99f8ff9700 1 -- 172.21.15.42:0/3883335558 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9a0c025d40 con 0x55abc5577380 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f99f8ff9700 1 -- 172.21.15.42:0/3883335558 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a0c025820 con 0x55abc5577380 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a23a10b80 10 client.15462 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3883335558 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55abc531b200 con 0x55abc5577380 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f99f8ff9700 1 -- 172.21.15.42:0/3883335558 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a0009b900 con 0x55abc5577380 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f99f8ff9700 1 -- 172.21.15.42:0/3883335558 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f9a0c02d070 con 0x55abc5577380 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a23a10b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:22.329 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a23a10b80 10 client.15462 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a23a10b80 20 client.15462 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55abc558ca90 0x55abc55e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a23a10b80 4 client.15462 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3883335558 --> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55abc5491f80 con 0x55abc558ca90 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.320+0000 7f9a14db2700 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55abc558ca90 0x55abc55e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a14db2700 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55abc558ca90 0x55abc55e39e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2055ef85529b331 server_cookie=11203e38f3297ab2 in_seq=0 out_seq=0 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f99f8ff9700 10 client.15462 ms_handle_connect on v2:172.21.15.173:6836/1667746348 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f99f8ff9700 1 -- 172.21.15.42:0/3883335558 <== mds.2 v2:172.21.15.173:6836/1667746348 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55abc5491f80 con 0x55abc558ca90 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f99f8ff9700 10 client.15462 handle_command_reply: tid=0 2023-12-12T09:58:22.330 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 client.15462 shutdown 2023-12-12T09:58:22.331 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3883335558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 msgr2=0x55abc5587b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.331 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5587b80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c000c00 tx=0x7f9a0c012000 comp rx=0 tx=0).stop 2023-12-12T09:58:22.331 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3883335558 shutdown_connections 2023-12-12T09:58:22.331 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.173:6836/1667746348,v1:172.21.15.173:6837/1667746348] conn(0x55abc558ca90 0x55abc55e39e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.331 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55abc5578bd0 0x55abc55846c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.331 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55abc5577380 0x55abc5587b80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.331 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 --2- 172.21.15.42:0/3883335558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55abc5577ce0 0x55abc55880c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3883335558 >> 172.21.15.42:0/3883335558 conn(0x55abc54a6790 msgr2=0x55abc557ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f99f8ff9700 0 client.15462 ms_handle_reset on v2:172.21.15.173:6836/1667746348 2023-12-12T09:58:22.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3883335558 shutdown_connections 2023-12-12T09:58:22.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 1 -- 172.21.15.42:0/3883335558 wait complete. 2023-12-12T09:58:22.332 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.321+0000 7f9a23a10b80 20 client.15462 trim_cache size 0 max 16384 2023-12-12T09:58:22.332 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:58:22.332 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:58:22.332 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2023-12-12T09:58:22.332 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24487, 2023-12-12T09:58:22.333 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:58:22.333 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:58:22.333 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:58:22.333 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 78.044135389000004, 2023-12-12T09:58:22.333 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:58:22.333 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:58:22.333 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:58:22.333 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 121.541570814 2023-12-12T09:58:22.333 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:58:22.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 -- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99fc0a1510 msgr2=0x7f99fc0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99fc0a1510 0x7f99fc0a39d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9a101014d0 tx=0x7f9a0c00a3b0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 -- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 msgr2=0x7f9a10199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 0x7f9a10199bc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9a00009fd0 tx=0x7f9a00005f10 comp rx=0 tx=0).stop 2023-12-12T09:58:22.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 -- 172.21.15.42:0/2964988194 shutdown_connections 2023-12-12T09:58:22.334 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f99fc0a1510 0x7f99fc0a39d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9a10100480 0x7f9a10199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a10102010 0x7f9a101a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 --2- 172.21.15.42:0/2964988194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9a10101670 0x7f9a10199bc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 -- 172.21.15.42:0/2964988194 >> 172.21.15.42:0/2964988194 conn(0x7f9a100fb9f0 msgr2=0x7f9a10068610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 -- 172.21.15.42:0/2964988194 shutdown_connections 2023-12-12T09:58:22.335 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.322+0000 7f9a17817700 1 -- 172.21.15.42:0/2964988194 wait complete. 2023-12-12T09:58:22.336 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.461+0000 7fc9683be700 1 -- 172.21.15.42:0/360515518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960100480 msgr2=0x7fc9601008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.461+0000 7fc9683be700 1 --2- 172.21.15.42:0/360515518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960100480 0x7fc9601008a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0099c0 tx=0x7fc95c009cd0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.461+0000 7fc9683be700 1 -- 172.21.15.42:0/360515518 shutdown_connections 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.461+0000 7fc9683be700 1 --2- 172.21.15.42:0/360515518 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc960102010 0x7fc9601088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.461+0000 7fc9683be700 1 --2- 172.21.15.42:0/360515518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960100480 0x7fc9601008a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.461+0000 7fc9683be700 1 --2- 172.21.15.42:0/360515518 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc960101670 0x7fc960101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.461+0000 7fc9683be700 1 -- 172.21.15.42:0/360515518 >> 172.21.15.42:0/360515518 conn(0x7fc9600fb9f0 msgr2=0x7fc9600fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 -- 172.21.15.42:0/360515518 shutdown_connections 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 -- 172.21.15.42:0/360515518 wait complete. 2023-12-12T09:58:22.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 Processor -- start 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 -- start start 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc960100480 0x7fc960197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc960101670 0x7fc960197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102010 0x7fc96019ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96019f060 con 0x7fc960102010 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96019f1a0 con 0x7fc960100480 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.462+0000 7fc9683be700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96019f330 con 0x7fc960101670 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96695b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102010 0x7fc96019ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96695b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102010 0x7fc96019ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35270/0 (socket says 172.21.15.42:35270) 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96695b700 1 -- 172.21.15.42:0/2384428067 learned_addr learned my addr 172.21.15.42:0/2384428067 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc965959700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc960101670 0x7fc960197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96615a700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc960100480 0x7fc960197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96695b700 1 -- 172.21.15.42:0/2384428067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc960101670 msgr2=0x7fc960197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96695b700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc960101670 0x7fc960197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96695b700 1 -- 172.21.15.42:0/2384428067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc960100480 msgr2=0x7fc960197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96695b700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc960100480 0x7fc960197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96695b700 1 -- 172.21.15.42:0/2384428067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95c0095f0 con 0x7fc960102010 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc965959700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc960101670 0x7fc960197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc96695b700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102010 0x7fc96019ea90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc958007d70 tx=0x7fc95800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc9537fe700 1 -- 172.21.15.42:0/2384428067 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95800ad40 con 0x7fc960102010 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc9537fe700 1 -- 172.21.15.42:0/2384428067 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9580128e0 con 0x7fc960102010 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc9683be700 1 -- 172.21.15.42:0/2384428067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96019f610 con 0x7fc960102010 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc9537fe700 1 -- 172.21.15.42:0/2384428067 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958011780 con 0x7fc960102010 2023-12-12T09:58:22.468 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.463+0000 7fc9683be700 1 -- 172.21.15.42:0/2384428067 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96019fb20 con 0x7fc960102010 2023-12-12T09:58:22.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.466+0000 7fc9537fe700 1 -- 172.21.15.42:0/2384428067 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7fc958012a50 con 0x7fc960102010 2023-12-12T09:58:22.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.466+0000 7fc9537fe700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc94c0a1510 0x7fc94c0a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.466+0000 7fc9537fe700 1 -- 172.21.15.42:0/2384428067 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9580cd3c0 con 0x7fc960102010 2023-12-12T09:58:22.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.466+0000 7fc96615a700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc94c0a1510 0x7fc94c0a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc9745b7b80 1 Processor -- start 2023-12-12T09:58:22.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc96615a700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc94c0a1510 0x7fc94c0a39d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009990 tx=0x7fc95c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.474 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc9745b7b80 1 -- start start 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 0x55b933c345f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 0x55b933c35050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933b8eb00 con 0x55b933c33380 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933b8c420 con 0x55b933c33ce0 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933b87d10 con 0x55b933c34bd0 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc96615a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc96615a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35274/0 (socket says 172.21.15.42:35274) 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.467+0000 7fc96615a700 1 -- 172.21.15.42:0/3277402742 learned_addr learned my addr 172.21.15.42:0/3277402742 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc965959700 1 --2- 172.21.15.42:0/3277402742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 0x55b933c345f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc96695b700 1 --2- 172.21.15.42:0/3277402742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 0x55b933c35050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc96615a700 1 -- 172.21.15.42:0/3277402742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 msgr2=0x55b933c35050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc96615a700 1 --2- 172.21.15.42:0/3277402742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 0x55b933c35050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc96615a700 1 -- 172.21.15.42:0/3277402742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 msgr2=0x55b933c345f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.475 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc96615a700 1 --2- 172.21.15.42:0/3277402742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 0x55b933c345f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc96615a700 1 -- 172.21.15.42:0/3277402742 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b933a045e0 con 0x55b933c33380 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc96615a700 1 --2- 172.21.15.42:0/3277402742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c337a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc9601014d0 tx=0x7fc95c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7a5a293582b508 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc950d82700 1 -- 172.21.15.42:0/3277402742 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c01d040 con 0x55b933c33380 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc950d82700 1 -- 172.21.15.42:0/3277402742 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc95c029040 con 0x55b933c33380 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.468+0000 7fc950d82700 1 -- 172.21.15.42:0/3277402742 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c023d90 con 0x55b933c33380 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3277402742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 msgr2=0x55b933c337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3277402742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c337a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc9601014d0 tx=0x7fc95c01e000 comp rx=0 tx=0).stop 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3277402742 shutdown_connections 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3277402742 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 0x55b933c35050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3277402742 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c337a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3277402742 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 0x55b933c345f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3277402742 >> 172.21.15.42:0/3277402742 conn(0x55b933b62790 msgr2=0x55b933c2fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3277402742 shutdown_connections 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3277402742 wait complete. 2023-12-12T09:58:22.476 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 -- start start 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.469+0000 7fc9745b7b80 1 Processor -- start 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33380 0x55b933c3d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33ce0 0x55b933c3d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c34bd0 0x55b933c446e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933b41510 con 0x55b933c34bd0 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933b411b0 con 0x55b933c33380 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933a068d0 con 0x55b933c33ce0 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc9745b7b80 10 client.0 _ll_register_callbacks cb 0x55b933a6ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc96615a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33380 0x55b933c3d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc965959700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33ce0 0x55b933c3d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc96615a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33380 0x55b933c3d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:38340/0 (socket says 172.21.15.42:38340) 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc96615a700 1 -- 172.21.15.42:0/3741153187 learned_addr learned my addr 172.21.15.42:0/3741153187 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc96615a700 1 -- 172.21.15.42:0/3741153187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33ce0 msgr2=0x55b933c3d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc96615a700 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33ce0 0x55b933c3d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.477 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc96615a700 1 -- 172.21.15.42:0/3741153187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c34bd0 msgr2=0x55b933c446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc96615a700 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c34bd0 0x55b933c446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc96615a700 1 -- 172.21.15.42:0/3741153187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc960109210 con 0x55b933c33380 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.470+0000 7fc965959700 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33ce0 0x55b933c3d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc96615a700 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33380 0x55b933c3d160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00a010 tx=0x7fc95c024e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d1e4dbc0cc8ae8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc949ffb700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc949ffb700 1 -- 172.21.15.42:0/3741153187 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c00fd10 con 0x55b933c33380 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3741153187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b933c26af0 con 0x55b933c33380 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc949ffb700 1 -- 172.21.15.42:0/3741153187 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc95c03c070 con 0x55b933c33380 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc949ffb700 1 -- 172.21.15.42:0/3741153187 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc95c00fe70 con 0x55b933c33380 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc9745b7b80 10 client.25196 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3741153187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b9339b0d90 con 0x55b933c33380 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc949ffb700 1 -- 172.21.15.42:0/3741153187 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b933c26af0 con 0x55b933c33380 2023-12-12T09:58:22.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc949ffb700 1 -- 172.21.15.42:0/3741153187 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc95c02c070 con 0x55b933c33380 2023-12-12T09:58:22.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc9745b7b80 10 client.25196 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:22.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc9745b7b80 10 client.25196 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-12T09:58:22.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc9745b7b80 20 client.25196 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:22.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x55b933c488f0 0x55b933c476b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc9745b7b80 4 client.25196 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:58:22.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.471+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3741153187 --> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b933c02ef0 con 0x55b933c488f0 2023-12-12T09:58:22.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.472+0000 7fc965959700 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x55b933c488f0 0x55b933c476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.472+0000 7fc965959700 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x55b933c488f0 0x55b933c476b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab2c1b690ccaee1 server_cookie=c5b0e96dac99dd6c in_seq=0 out_seq=0 2023-12-12T09:58:22.479 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.472+0000 7fc949ffb700 10 client.25196 ms_handle_connect on v2:172.21.15.173:6838/2977372273 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc949ffb700 1 -- 172.21.15.42:0/3741153187 <== mds.1 v2:172.21.15.173:6838/2977372273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b933c02ef0 con 0x55b933c488f0 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc949ffb700 10 client.25196 handle_command_reply: tid=0 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 client.25196 shutdown 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3741153187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33380 msgr2=0x55b933c3d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33380 0x55b933c3d160 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc95c00a010 tx=0x7fc95c024e50 comp rx=0 tx=0).stop 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3741153187 shutdown_connections 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33380 0x55b933c3d160 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x55b933c488f0 0x55b933c476b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c34bd0 0x55b933c446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc949ffb700 0 client.25196 ms_handle_reset on v2:172.21.15.173:6838/2977372273 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/3741153187 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33ce0 0x55b933c3d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3741153187 >> 172.21.15.42:0/3741153187 conn(0x55b933b62790 msgr2=0x55b933c2e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.481 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3741153187 shutdown_connections 2023-12-12T09:58:22.482 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 1 -- 172.21.15.42:0/3741153187 wait complete. 2023-12-12T09:58:22.482 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.473+0000 7fc9745b7b80 20 client.25196 trim_cache size 0 max 16384 2023-12-12T09:58:22.496 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc9745b7b80 1 Processor -- start 2023-12-12T09:58:22.496 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc9745b7b80 1 -- start start 2023-12-12T09:58:22.496 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33380 0x55b933c44d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33ce0 0x55b933c44670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c34bd0 0x55b933c51250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933b88fa0 con 0x55b933c33ce0 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933b83e00 con 0x55b933c34bd0 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933b2cb50 con 0x55b933c33380 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc965959700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33ce0 0x55b933c44670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc965959700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33ce0 0x55b933c44670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35290/0 (socket says 172.21.15.42:35290) 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.491+0000 7fc965959700 1 -- 172.21.15.42:0/1394676207 learned_addr learned my addr 172.21.15.42:0/1394676207 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc96695b700 1 --2- 172.21.15.42:0/1394676207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c34bd0 0x55b933c51250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc96615a700 1 --2- 172.21.15.42:0/1394676207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33380 0x55b933c44d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc965959700 1 -- 172.21.15.42:0/1394676207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33380 msgr2=0x55b933c44d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc965959700 1 --2- 172.21.15.42:0/1394676207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33380 0x55b933c44d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.497 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc965959700 1 -- 172.21.15.42:0/1394676207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c34bd0 msgr2=0x55b933c51250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc965959700 1 --2- 172.21.15.42:0/1394676207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c34bd0 0x55b933c51250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc965959700 1 -- 172.21.15.42:0/1394676207 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b933999c10 con 0x55b933c33ce0 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc965959700 1 --2- 172.21.15.42:0/1394676207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33ce0 0x55b933c44670 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc954005d90 tx=0x7fc954009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53ed9faedba30d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc951583700 1 -- 172.21.15.42:0/1394676207 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954018040 con 0x55b933c33ce0 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc951583700 1 -- 172.21.15.42:0/1394676207 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc95401a040 con 0x55b933c33ce0 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.492+0000 7fc951583700 1 -- 172.21.15.42:0/1394676207 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc954003be0 con 0x55b933c33ce0 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 -- 172.21.15.42:0/1394676207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33ce0 msgr2=0x55b933c44670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/1394676207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33ce0 0x55b933c44670 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc954005d90 tx=0x7fc954009fd0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 -- 172.21.15.42:0/1394676207 shutdown_connections 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/1394676207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c34bd0 0x55b933c51250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/1394676207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c33380 0x55b933c44d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.498 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/1394676207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33ce0 0x55b933c44670 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 -- 172.21.15.42:0/1394676207 >> 172.21.15.42:0/1394676207 conn(0x55b933b62790 msgr2=0x55b933c3c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 -- 172.21.15.42:0/1394676207 shutdown_connections 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 -- 172.21.15.42:0/1394676207 wait complete. 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 -- start start 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 Processor -- start 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c405c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 0x55b933c40b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 0x55b933c43b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933c44060 con 0x55b933c33380 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933c441d0 con 0x55b933c33ce0 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b933c44390 con 0x55b933c34bd0 2023-12-12T09:58:22.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc9745b7b80 10 client.0 _ll_register_callbacks cb 0x55b933a6ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc96615a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc96615a700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c405c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35294/0 (socket says 172.21.15.42:35294) 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc96615a700 1 -- 172.21.15.42:0/2559647263 learned_addr learned my addr 172.21.15.42:0/2559647263 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.493+0000 7fc965959700 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 0x55b933c40b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.494+0000 7fc96695b700 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 0x55b933c43b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.494+0000 7fc96615a700 1 -- 172.21.15.42:0/2559647263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 msgr2=0x55b933c43b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.494+0000 7fc96615a700 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 0x55b933c43b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.494+0000 7fc96615a700 1 -- 172.21.15.42:0/2559647263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 msgr2=0x55b933c40b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.494+0000 7fc96615a700 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 0x55b933c40b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.494+0000 7fc96615a700 1 -- 172.21.15.42:0/2559647263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc954009e40 con 0x55b933c33380 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.494+0000 7fc96695b700 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 0x55b933c43b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.494+0000 7fc96615a700 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c405c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0365a0 tx=0x7fc95c036420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5ec5d646a2544f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.494+0000 7fc949ffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:22.502 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.496+0000 7fc949ffb700 1 -- 172.21.15.42:0/2559647263 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c01d040 con 0x55b933c33380 2023-12-12T09:58:22.502 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.496+0000 7fc949ffb700 1 -- 172.21.15.42:0/2559647263 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc95c03c070 con 0x55b933c33380 2023-12-12T09:58:22.502 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.496+0000 7fc9745b7b80 1 -- 172.21.15.42:0/2559647263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc95c029040 con 0x55b933c33380 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.496+0000 7fc949ffb700 1 -- 172.21.15.42:0/2559647263 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc95c036920 con 0x55b933c33380 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.496+0000 7fc9745b7b80 10 client.15477 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.496+0000 7fc9745b7b80 1 -- 172.21.15.42:0/2559647263 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b9339d7200 con 0x55b933c33380 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.497+0000 7fc949ffb700 1 -- 172.21.15.42:0/2559647263 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc95c029040 con 0x55b933c33380 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.497+0000 7fc949ffb700 1 -- 172.21.15.42:0/2559647263 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc95c00f460 con 0x55b933c33380 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.497+0000 7fc9745b7b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.497+0000 7fc9745b7b80 10 client.15477 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.497+0000 7fc9745b7b80 20 client.15477 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.497+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x55b933ca3160 0x55b933c9f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.497+0000 7fc9745b7b80 4 client.15477 mds_command: new command op to 24508 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.497+0000 7fc9745b7b80 1 -- 172.21.15.42:0/2559647263 --> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b933b4df80 con 0x55b933ca3160 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.497+0000 7fc965959700 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x55b933ca3160 0x55b933c9f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.503 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc965959700 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x55b933ca3160 0x55b933c9f5d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=faf3c3d447b02c7a server_cookie=5608ad054f0d32f0 in_seq=0 out_seq=0 2023-12-12T09:58:22.504 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc949ffb700 10 client.15477 ms_handle_connect on v2:172.21.15.173:6838/2977372273 2023-12-12T09:58:22.504 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc949ffb700 1 -- 172.21.15.42:0/2559647263 <== mds.1 v2:172.21.15.173:6838/2977372273 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b933b4df80 con 0x55b933ca3160 2023-12-12T09:58:22.504 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc949ffb700 10 client.15477 handle_command_reply: tid=0 2023-12-12T09:58:22.504 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 client.15477 shutdown 2023-12-12T09:58:22.504 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 -- 172.21.15.42:0/2559647263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 msgr2=0x55b933c405c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.504 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c405c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0365a0 tx=0x7fc95c036420 comp rx=0 tx=0).stop 2023-12-12T09:58:22.504 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:22 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1193926989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:58:22.505 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 -- 172.21.15.42:0/2559647263 shutdown_connections 2023-12-12T09:58:22.505 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.173:6838/2977372273,v1:172.21.15.173:6839/2977372273] conn(0x55b933ca3160 0x55b933c9f5d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.505 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b933c34bd0 0x55b933c43b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.505 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc949ffb700 0 client.15477 ms_handle_reset on v2:172.21.15.173:6838/2977372273 2023-12-12T09:58:22.505 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b933c33380 0x55b933c405c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.505 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 --2- 172.21.15.42:0/2559647263 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b933c33ce0 0x55b933c40b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.505 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 -- 172.21.15.42:0/2559647263 >> 172.21.15.42:0/2559647263 conn(0x55b933b62790 msgr2=0x55b933c51250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 -- 172.21.15.42:0/2559647263 shutdown_connections 2023-12-12T09:58:22.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.498+0000 7fc9745b7b80 1 -- 172.21.15.42:0/2559647263 wait complete. 2023-12-12T09:58:22.506 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9745b7b80 20 client.15477 trim_cache size 0 max 16384 2023-12-12T09:58:22.506 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:58:22.506 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:58:22.506 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2023-12-12T09:58:22.506 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24508, 2023-12-12T09:58:22.506 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:58:22.506 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:58:22.507 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:58:22.507 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 79.231785479999999, 2023-12-12T09:58:22.507 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:58:22.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:58:22.507 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:58:22.507 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 117.286035588 2023-12-12T09:58:22.507 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:58:22.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 -- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc94c0a1510 msgr2=0x7fc94c0a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc94c0a1510 0x7fc94c0a39d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc95c009990 tx=0x7fc95c01a040 comp rx=0 tx=0).stop 2023-12-12T09:58:22.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 -- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102010 msgr2=0x7fc96019ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102010 0x7fc96019ea90 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc958007d70 tx=0x7fc95800a340 comp rx=0 tx=0).stop 2023-12-12T09:58:22.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 -- 172.21.15.42:0/2384428067 shutdown_connections 2023-12-12T09:58:22.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7fc94c0a1510 0x7fc94c0a39d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc960100480 0x7fc960197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc960102010 0x7fc96019ea90 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.508 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 --2- 172.21.15.42:0/2384428067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc960101670 0x7fc960197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.509 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 -- 172.21.15.42:0/2384428067 >> 172.21.15.42:0/2384428067 conn(0x7fc9600fb9f0 msgr2=0x7fc9600fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.509 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 -- 172.21.15.42:0/2384428067 shutdown_connections 2023-12-12T09:58:22.509 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.499+0000 7fc9683be700 1 -- 172.21.15.42:0/2384428067 wait complete. 2023-12-12T09:58:22.513 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-12T09:58:22.640 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.638+0000 7efe89007700 1 -- 172.21.15.42:0/2011712756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 msgr2=0x7efe841099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.638+0000 7efe89007700 1 --2- 172.21.15.42:0/2011712756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 0x7efe841099e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00b0d0 tx=0x7efe7c00b3e0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.638+0000 7efe89007700 1 -- 172.21.15.42:0/2011712756 shutdown_connections 2023-12-12T09:58:22.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.638+0000 7efe89007700 1 --2- 172.21.15.42:0/2011712756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe841015a0 0x7efe841019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.638+0000 7efe89007700 1 --2- 172.21.15.42:0/2011712756 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 0x7efe841099e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.638+0000 7efe89007700 1 --2- 172.21.15.42:0/2011712756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe84102790 0x7efe84102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.638+0000 7efe89007700 1 -- 172.21.15.42:0/2011712756 >> 172.21.15.42:0/2011712756 conn(0x7efe840fcb10 msgr2=0x7efe840fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.638+0000 7efe89007700 1 -- 172.21.15.42:0/2011712756 shutdown_connections 2023-12-12T09:58:22.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.638+0000 7efe89007700 1 -- 172.21.15.42:0/2011712756 wait complete. 2023-12-12T09:58:22.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe89007700 1 Processor -- start 2023-12-12T09:58:22.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe89007700 1 -- start start 2023-12-12T09:58:22.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe89007700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe841015a0 0x7efe84197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe89007700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102790 0x7efe84197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.643 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe89007700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 0x7efe8419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe89007700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8419f0b0 con 0x7efe84103130 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe89007700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8419f1f0 con 0x7efe84102790 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe89007700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8419f380 con 0x7efe841015a0 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe8359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 0x7efe8419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe8359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 0x7efe8419eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35312/0 (socket says 172.21.15.42:35312) 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.639+0000 7efe8359e700 1 -- 172.21.15.42:0/1299624980 learned_addr learned my addr 172.21.15.42:0/1299624980 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe841015a0 0x7efe84197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe8259c700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102790 0x7efe84197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe8359e700 1 -- 172.21.15.42:0/1299624980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe841015a0 msgr2=0x7efe84197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe8359e700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe841015a0 0x7efe84197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe8359e700 1 -- 172.21.15.42:0/1299624980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102790 msgr2=0x7efe84197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe8359e700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102790 0x7efe84197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe8359e700 1 -- 172.21.15.42:0/1299624980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c009d00 con 0x7efe84103130 2023-12-12T09:58:22.644 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe841015a0 0x7efe84197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:22.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe8259c700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102790 0x7efe84197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:22.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe8359e700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 0x7efe8419eae0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efe7c007bd0 tx=0x7efe7c0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe7bfff700 1 -- 172.21.15.42:0/1299624980 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c00e040 con 0x7efe84103130 2023-12-12T09:58:22.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe89007700 1 -- 172.21.15.42:0/1299624980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8419f600 con 0x7efe84103130 2023-12-12T09:58:22.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe7bfff700 1 -- 172.21.15.42:0/1299624980 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe7c012dc0 con 0x7efe84103130 2023-12-12T09:58:22.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe7bfff700 1 -- 172.21.15.42:0/1299624980 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c01b450 con 0x7efe84103130 2023-12-12T09:58:22.645 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.640+0000 7efe89007700 1 -- 172.21.15.42:0/1299624980 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8419fad0 con 0x7efe84103130 2023-12-12T09:58:22.651 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.642+0000 7efe7bfff700 1 -- 172.21.15.42:0/1299624980 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7efe7c01d610 con 0x7efe84103130 2023-12-12T09:58:22.651 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.643+0000 7efe7bfff700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efe700a1590 0x7efe700a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.651 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.643+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efe700a1590 0x7efe700a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.651 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.643+0000 7efe7bfff700 1 -- 172.21.15.42:0/1299624980 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efe7c0cef90 con 0x7efe84103130 2023-12-12T09:58:22.651 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efe700a1590 0x7efe700a3a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efe74005fd0 tx=0x7efe74005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe95200b80 1 Processor -- start 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe95200b80 1 -- start start 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe95200b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7cf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe95200b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe95200b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe95200b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe72ab00 con 0x55b4fe7cf380 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe95200b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe728420 con 0x55b4fe7cfce0 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe95200b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe723d10 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.644+0000 7efe82d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7cf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe82d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7cf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35316/0 (socket says 172.21.15.42:35316) 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe82d9d700 1 -- 172.21.15.42:0/1727411823 learned_addr learned my addr 172.21.15.42:0/1727411823 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe8359e700 1 --2- 172.21.15.42:0/1727411823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe8259c700 1 --2- 172.21.15.42:0/1727411823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe82d9d700 1 -- 172.21.15.42:0/1727411823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7d0bd0 msgr2=0x55b4fe7d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.652 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1727411823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7d1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe82d9d700 1 -- 172.21.15.42:0/1727411823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cfce0 msgr2=0x55b4fe7d05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1727411823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe82d9d700 1 -- 172.21.15.42:0/1727411823 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4fe5a05e0 con 0x55b4fe7cf380 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1727411823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7cf7a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efe7400edb0 tx=0x7efe74011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c687e122ee31af server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe797fa700 1 -- 172.21.15.42:0/1727411823 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74011c90 con 0x55b4fe7cf380 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe797fa700 1 -- 172.21.15.42:0/1727411823 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe74011df0 con 0x55b4fe7cf380 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.645+0000 7efe797fa700 1 -- 172.21.15.42:0/1727411823 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74010750 con 0x55b4fe7cf380 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 -- 172.21.15.42:0/1727411823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 msgr2=0x55b4fe7cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 --2- 172.21.15.42:0/1727411823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7cf7a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efe7400edb0 tx=0x7efe74011000 comp rx=0 tx=0).stop 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 -- 172.21.15.42:0/1727411823 shutdown_connections 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 --2- 172.21.15.42:0/1727411823 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 --2- 172.21.15.42:0/1727411823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7cf7a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 --2- 172.21.15.42:0/1727411823 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 -- 172.21.15.42:0/1727411823 >> 172.21.15.42:0/1727411823 conn(0x55b4fe6fe790 msgr2=0x55b4fe7cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.653 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 -- 172.21.15.42:0/1727411823 shutdown_connections 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 -- 172.21.15.42:0/1727411823 wait complete. 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 -- start start 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.646+0000 7efe95200b80 1 Processor -- start 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe95200b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe95200b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe95200b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe95200b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe6dd510 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe95200b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe6dd1b0 con 0x55b4fe7cf380 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe95200b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe5a28d0 con 0x55b4fe7cfce0 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe95200b80 10 client.0 _ll_register_callbacks cb 0x55b4fe608a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35326/0 (socket says 172.21.15.42:35326) 2023-12-12T09:58:22.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8359e700 1 -- 172.21.15.42:0/1701209611 learned_addr learned my addr 172.21.15.42:0/1701209611 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8259c700 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8359e700 1 -- 172.21.15.42:0/1701209611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 msgr2=0x55b4fe7d9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8359e700 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8359e700 1 -- 172.21.15.42:0/1701209611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cf380 msgr2=0x55b4fe7d9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8359e700 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8359e700 1 -- 172.21.15.42:0/1701209611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7400ea90 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe8359e700 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e06b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efe7c098940 tx=0x7efe7c09bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f7415041e6d0f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe6a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe6a7fc700 1 -- 172.21.15.42:0/1701209611 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c09bed0 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe6a7fc700 1 -- 172.21.15.42:0/1701209611 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efe7c0a0020 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.647+0000 7efe95200b80 1 -- 172.21.15.42:0/1701209611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4fe7c2af0 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe6a7fc700 1 -- 172.21.15.42:0/1701209611 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe7c0a41d0 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe95200b80 10 client.15489 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe95200b80 1 -- 172.21.15.42:0/1701209611 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b4fe54cd90 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe6a7fc700 1 -- 172.21.15.42:0/1701209611 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe7c07e080 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe6a7fc700 1 -- 172.21.15.42:0/1701209611 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efe7c0a3d40 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe95200b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe95200b80 10 client.15489 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe95200b80 20 client.15489 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe95200b80 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b4fe7e4af0 0x55b4fe7e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe95200b80 4 client.15489 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.648+0000 7efe95200b80 1 -- 172.21.15.42:0/1701209611 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4fe79eef0 con 0x55b4fe7e4af0 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.649+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b4fe7e4af0 0x55b4fe7e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.649+0000 7efe82d9d700 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b4fe7e4af0 0x55b4fe7e36d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a3ca52bbe308ff server_cookie=f640fec07129f3e0 in_seq=0 out_seq=0 2023-12-12T09:58:22.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.649+0000 7efe6a7fc700 10 client.15489 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe6a7fc700 1 -- 172.21.15.42:0/1701209611 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b4fe79eef0 con 0x55b4fe7e4af0 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe6a7fc700 10 client.15489 handle_command_reply: tid=0 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 client.15489 shutdown 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 -- 172.21.15.42:0/1701209611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7d0bd0 msgr2=0x55b4fe7e06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e06b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efe7c098940 tx=0x7efe7c09bb00 comp rx=0 tx=0).stop 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 -- 172.21.15.42:0/1701209611 shutdown_connections 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b4fe7e4af0 0x55b4fe7e36d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e06b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 --2- 172.21.15.42:0/1701209611 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 -- 172.21.15.42:0/1701209611 >> 172.21.15.42:0/1701209611 conn(0x55b4fe6fe790 msgr2=0x55b4fe7ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe6a7fc700 0 client.15489 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:58:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 -- 172.21.15.42:0/1701209611 shutdown_connections 2023-12-12T09:58:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 1 -- 172.21.15.42:0/1701209611 wait complete. 2023-12-12T09:58:22.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.650+0000 7efe95200b80 20 client.15489 trim_cache size 0 max 16384 2023-12-12T09:58:22.671 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.668+0000 7efe95200b80 1 Processor -- start 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.668+0000 7efe95200b80 1 -- start start 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe95200b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe95200b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe95200b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe95200b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe71fe00 con 0x55b4fe7cf380 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe95200b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe6c8b50 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe95200b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe6c6830 con 0x55b4fe7cfce0 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe82d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe82d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35330/0 (socket says 172.21.15.42:35330) 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe82d9d700 1 -- 172.21.15.42:0/3618271411 learned_addr learned my addr 172.21.15.42:0/3618271411 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe8259c700 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe8359e700 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.672 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe82d9d700 1 -- 172.21.15.42:0/3618271411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 msgr2=0x55b4fe7d40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe82d9d700 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe82d9d700 1 -- 172.21.15.42:0/3618271411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 msgr2=0x55b4fe7e0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe82d9d700 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe82d9d700 1 -- 172.21.15.42:0/3618271411 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4fe535c10 con 0x55b4fe7cf380 2023-12-12T09:58:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe8259c700 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe82d9d700 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d3bb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efe74011fd0 tx=0x7efe74011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc2593eadda8065 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe79ffb700 1 -- 172.21.15.42:0/3618271411 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe74011c90 con 0x55b4fe7cf380 2023-12-12T09:58:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe79ffb700 1 -- 172.21.15.42:0/3618271411 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe74030d70 con 0x55b4fe7cf380 2023-12-12T09:58:22.673 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.669+0000 7efe79ffb700 1 -- 172.21.15.42:0/3618271411 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe740046f0 con 0x55b4fe7cf380 2023-12-12T09:58:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.670+0000 7efe95200b80 1 -- 172.21.15.42:0/3618271411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 msgr2=0x55b4fe7d3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.670+0000 7efe95200b80 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d3bb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efe74011fd0 tx=0x7efe74011f60 comp rx=0 tx=0).stop 2023-12-12T09:58:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.670+0000 7efe95200b80 1 -- 172.21.15.42:0/3618271411 shutdown_connections 2023-12-12T09:58:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.670+0000 7efe95200b80 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7e0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.670+0000 7efe95200b80 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cf380 0x55b4fe7d3bb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.670+0000 7efe95200b80 1 --2- 172.21.15.42:0/3618271411 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.670+0000 7efe95200b80 1 -- 172.21.15.42:0/3618271411 >> 172.21.15.42:0/3618271411 conn(0x55b4fe6fe790 msgr2=0x55b4fe7d7190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.670+0000 7efe95200b80 1 -- 172.21.15.42:0/3618271411 shutdown_connections 2023-12-12T09:58:22.678 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.670+0000 7efe95200b80 1 -- 172.21.15.42:0/3618271411 wait complete. 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe95200b80 1 -- start start 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe95200b80 1 Processor -- start 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe95200b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cf380 0x55b4fe7e0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe95200b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe95200b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe95200b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe7ddbf0 con 0x55b4fe7cfce0 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe95200b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe83abb0 con 0x55b4fe7d0bd0 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe95200b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4fe83acf0 con 0x55b4fe7cf380 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe95200b80 10 client.0 _ll_register_callbacks cb 0x55b4fe608a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7dd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35338/0 (socket says 172.21.15.42:35338) 2023-12-12T09:58:22.679 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8259c700 1 -- 172.21.15.42:0/18243392 learned_addr learned my addr 172.21.15.42:0/18243392 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe82d9d700 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cf380 0x55b4fe7e0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8359e700 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8259c700 1 -- 172.21.15.42:0/18243392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cf380 msgr2=0x55b4fe7e0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8259c700 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cf380 0x55b4fe7e0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8259c700 1 -- 172.21.15.42:0/18243392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 msgr2=0x55b4fe7dd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8259c700 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7dd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8259c700 1 -- 172.21.15.42:0/18243392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4fe5a05e0 con 0x55b4fe7cfce0 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe8259c700 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7dd090 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efe6c011a60 tx=0x7efe6c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4511029eab432ac2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.671+0000 7efe6a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe6a7fc700 1 -- 172.21.15.42:0/18243392 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6c016070 con 0x55b4fe7cfce0 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe95200b80 1 -- 172.21.15.42:0/18243392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe74011df0 con 0x55b4fe7cfce0 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe6a7fc700 1 -- 172.21.15.42:0/18243392 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efe6c00e800 con 0x55b4fe7cfce0 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe6a7fc700 1 -- 172.21.15.42:0/18243392 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe6c0112e0 con 0x55b4fe7cfce0 2023-12-12T09:58:22.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe95200b80 10 client.15495 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe95200b80 1 -- 172.21.15.42:0/18243392 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b4fe573200 con 0x55b4fe7cfce0 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe6a7fc700 1 -- 172.21.15.42:0/18243392 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe74011df0 con 0x55b4fe7cfce0 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe6a7fc700 1 -- 172.21.15.42:0/18243392 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7efe6c01bba0 con 0x55b4fe7cfce0 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe95200b80 10 client.15495 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe95200b80 10 client.15495 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe95200b80 20 client.15495 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe95200b80 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b4fe840030 0x55b4fe83c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe95200b80 4 client.15495 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.672+0000 7efe95200b80 1 -- 172.21.15.42:0/18243392 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b4fe6e9f80 con 0x55b4fe840030 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe82d9d700 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b4fe840030 0x55b4fe83c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe6a7fc700 10 client.15495 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe82d9d700 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b4fe840030 0x55b4fe83c490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e1b971a524da44a server_cookie=1e11a60bb9144f9 in_seq=0 out_seq=0 2023-12-12T09:58:22.682 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe6a7fc700 1 -- 172.21.15.42:0/18243392 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55b4fe6e9f80 con 0x55b4fe840030 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe6a7fc700 10 client.15495 handle_command_reply: tid=0 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 client.15495 shutdown 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 -- 172.21.15.42:0/18243392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cfce0 msgr2=0x55b4fe7dd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7dd090 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7efe6c011a60 tx=0x7efe6c011e20 comp rx=0 tx=0).stop 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 -- 172.21.15.42:0/18243392 shutdown_connections 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b4fe840030 0x55b4fe83c490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b4fe7d0bd0 0x55b4fe7dd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b4fe7cf380 0x55b4fe7e0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe6a7fc700 0 client.15495 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 --2- 172.21.15.42:0/18243392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b4fe7cfce0 0x55b4fe7dd090 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 -- 172.21.15.42:0/18243392 >> 172.21.15.42:0/18243392 conn(0x55b4fe6fe790 msgr2=0x55b4fe7d3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 -- 172.21.15.42:0/18243392 shutdown_connections 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.673+0000 7efe95200b80 1 -- 172.21.15.42:0/18243392 wait complete. 2023-12-12T09:58:22.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe95200b80 20 client.15495 trim_cache size 0 max 16384 2023-12-12T09:58:22.684 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T09:58:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "968ccaec-98d3-11ee-95a3-87774f69a715", 2023-12-12T09:58:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2023-12-12T09:58:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24524, 2023-12-12T09:58:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2023-12-12T09:58:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2023-12-12T09:58:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2023-12-12T09:58:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 104.213453888, 2023-12-12T09:58:22.684 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 27, 2023-12-12T09:58:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 89, 2023-12-12T09:58:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 87, 2023-12-12T09:58:22.685 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 110.292614798 2023-12-12T09:58:22.685 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 -- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efe700a1590 msgr2=0x7efe700a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efe700a1590 0x7efe700a3a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efe74005fd0 tx=0x7efe74005f00 comp rx=0 tx=0).stop 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 -- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 msgr2=0x7efe8419eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 0x7efe8419eae0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efe7c007bd0 tx=0x7efe7c0094f0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 -- 172.21.15.42:0/1299624980 shutdown_connections 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7efe700a1590 0x7efe700a3a50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe84103130 0x7efe8419eae0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe841015a0 0x7efe84197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 --2- 172.21.15.42:0/1299624980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efe84102790 0x7efe84197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 -- 172.21.15.42:0/1299624980 >> 172.21.15.42:0/1299624980 conn(0x7efe840fcb10 msgr2=0x7efe84105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 -- 172.21.15.42:0/1299624980 shutdown_connections 2023-12-12T09:58:22.686 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:22.674+0000 7efe89007700 1 -- 172.21.15.42:0/1299624980 wait complete. 2023-12-12T09:58:22.688 INFO:teuthology.run_tasks:Running task kclient... 2023-12-12T09:58:22.705 INFO:tasks.kclient:Mounting kernel clients... 2023-12-12T09:58:22.705 INFO:tasks.kclient:config is {'client.0': {}} 2023-12-12T09:58:22.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:22.706 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2023-12-12T09:58:22.723 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-12-12T09:58:22.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:22.724 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-12T09:58:22.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:22.803 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2023-12-12T09:58:22.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:22 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1193926989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:58:22.907 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-12-12T09:58:22.907 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-12T09:58:22.907 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-12T09:58:22.907 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph fs ls 2023-12-12T09:58:22.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:22 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1193926989' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T09:58:23.510 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T09:58:23.768 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:23 smithi042 ceph-mon[98894]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:23.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:23 smithi173 ceph-mon[103308]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:23.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:23 smithi111 ceph-mon[102565]: pgmap v316: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:24.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.819+0000 7f296a3d4700 1 -- 172.21.15.42:0/3280359077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29641076f0 msgr2=0x7f2964107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:24.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.819+0000 7f296a3d4700 1 --2- 172.21.15.42:0/3280359077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29641076f0 0x7f2964107ad0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f294c009990 tx=0x7f294c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:24.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.819+0000 7f296a3d4700 1 -- 172.21.15.42:0/3280359077 shutdown_connections 2023-12-12T09:58:24.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.819+0000 7f296a3d4700 1 --2- 172.21.15.42:0/3280359077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2964102040 0x7f296410e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:24.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.819+0000 7f296a3d4700 1 --2- 172.21.15.42:0/3280359077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f29641076f0 0x7f2964107ad0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:24.823 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.819+0000 7f296a3d4700 1 --2- 172.21.15.42:0/3280359077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2964101680 0x7f2964101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:24.823 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.819+0000 7f296a3d4700 1 -- 172.21.15.42:0/3280359077 >> 172.21.15.42:0/3280359077 conn(0x7f29640787c0 msgr2=0x7f2964078bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:24.823 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.819+0000 7f296a3d4700 1 -- 172.21.15.42:0/3280359077 shutdown_connections 2023-12-12T09:58:24.823 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.819+0000 7f296a3d4700 1 -- 172.21.15.42:0/3280359077 wait complete. 2023-12-12T09:58:24.823 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f296a3d4700 1 Processor -- start 2023-12-12T09:58:24.823 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f296a3d4700 1 -- start start 2023-12-12T09:58:24.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f296a3d4700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2964101680 0x7f296419d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:24.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f296a3d4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2964102040 0x7f296419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:24.827 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f296a3d4700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29641076f0 0x7f2964197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:24.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f296a3d4700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964197d90 con 0x7f2964101680 2023-12-12T09:58:24.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f296a3d4700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964197f00 con 0x7f29641076f0 2023-12-12T09:58:24.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f296a3d4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29641980c0 con 0x7f2964102040 2023-12-12T09:58:24.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f2963fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2964101680 0x7f296419d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:24.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f2963fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2964101680 0x7f296419d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35350/0 (socket says 172.21.15.42:35350) 2023-12-12T09:58:24.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f2963fff700 1 -- 172.21.15.42:0/54053978 learned_addr learned my addr 172.21.15.42:0/54053978 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:24.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f29637fe700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2964102040 0x7f296419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:24.828 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f2963fff700 1 -- 172.21.15.42:0/54053978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2964102040 msgr2=0x7f296419db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:24.829 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f2963fff700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2964102040 0x7f296419db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:24.829 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f2963fff700 1 -- 172.21.15.42:0/54053978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29641076f0 msgr2=0x7f2964197730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:24.829 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.820+0000 7f2968971700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29641076f0 0x7f2964197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:24.829 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.821+0000 7f2963fff700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29641076f0 0x7f2964197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:24.829 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.821+0000 7f2963fff700 1 -- 172.21.15.42:0/54053978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c0095f0 con 0x7f2964101680 2023-12-12T09:58:24.829 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.821+0000 7f29637fe700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2964102040 0x7f296419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:24.830 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.821+0000 7f2963fff700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2964101680 0x7f296419d5f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f294c000c00 tx=0x7f294c00f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:24.830 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.821+0000 7f29617fa700 1 -- 172.21.15.42:0/54053978 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c01d070 con 0x7f2964101680 2023-12-12T09:58:24.830 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.821+0000 7f29617fa700 1 -- 172.21.15.42:0/54053978 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f294c00fde0 con 0x7f2964101680 2023-12-12T09:58:24.830 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.821+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2964198340 con 0x7f2964101680 2023-12-12T09:58:24.830 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.821+0000 7f29617fa700 1 -- 172.21.15.42:0/54053978 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c017770 con 0x7f2964101680 2023-12-12T09:58:24.830 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.821+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29641a45e0 con 0x7f2964101680 2023-12-12T09:58:24.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.822+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2964109b80 con 0x7f2964101680 2023-12-12T09:58:24.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.824+0000 7f29617fa700 1 -- 172.21.15.42:0/54053978 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f294c022aa0 con 0x7f2964101680 2023-12-12T09:58:24.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.824+0000 7f29617fa700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f29500a1510 0x7f29500a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:24.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.825+0000 7f29617fa700 1 -- 172.21.15.42:0/54053978 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f294c0ce8d0 con 0x7f2964101680 2023-12-12T09:58:24.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.825+0000 7f29637fe700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f29500a1510 0x7f29500a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:24.831 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.825+0000 7f29637fe700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f29500a1510 0x7f29500a39d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2954005fd0 tx=0x7f2954005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:24.832 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:24.828+0000 7f29617fa700 1 -- 172.21.15.42:0/54053978 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f294c097af0 con 0x7f2964101680 2023-12-12T09:58:25.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.071+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2964198bf0 con 0x7f2964101680 2023-12-12T09:58:25.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.072+0000 7f29617fa700 1 -- 172.21.15.42:0/54053978 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+88 (secure 0 0 0) 0x7f294c09b110 con 0x7f2964101680 2023-12-12T09:58:25.075 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f29500a1510 msgr2=0x7f29500a39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f29500a1510 0x7f29500a39d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2954005fd0 tx=0x7f2954005e20 comp rx=0 tx=0).stop 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2964101680 msgr2=0x7f296419d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2964101680 0x7f296419d5f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f294c000c00 tx=0x7f294c00f880 comp rx=0 tx=0).stop 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 shutdown_connections 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f29500a1510 0x7f29500a39d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29641076f0 0x7f2964197730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2964101680 0x7f296419d5f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 --2- 172.21.15.42:0/54053978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2964102040 0x7f296419db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 >> 172.21.15.42:0/54053978 conn(0x7f29640787c0 msgr2=0x7f2964100270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:25.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 shutdown_connections 2023-12-12T09:58:25.077 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:25.074+0000 7f296a3d4700 1 -- 172.21.15.42:0/54053978 wait complete. 2023-12-12T09:58:25.469 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-12T09:58:25.469 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-12T09:58:25.469 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2023-12-12T09:58:25.469 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-12T09:58:25.469 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-12T09:58:25.469 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-12T09:58:25.469 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-12T09:58:25.469 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-12T09:58:25.470 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:58:25.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:25.470 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2023-12-12T09:58:25.489 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-12T09:58:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-12T09:58:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2023-12-12T09:58:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:58:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2023-12-12T09:58:25.489 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41731sec preferred_lft 41731sec 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-12T09:58:25.490 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2023-12-12T09:58:25.491 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-12T09:58:25.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:25.491 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:58:25.491 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:58:25.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2023-12-12T09:58:25.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2023-12-12T09:58:25.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2023-12-12T09:58:25.491 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-12T09:58:25.491 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:58:25.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:25.613 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-12T09:58:25.641 INFO:teuthology.orchestra.run.smithi042.stdout:1 2023-12-12T09:58:25.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:25.643 DEBUG:teuthology.orchestra.run.smithi042:> route 2023-12-12T09:58:25.702 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2023-12-12T09:58:25.703 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-12T09:58:25.703 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-12T09:58:25.703 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-12T09:58:25.703 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-12T09:58:25.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:25.703 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:58:25.703 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:58:25.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-12T09:58:25.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-12T09:58:25.704 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-12T09:58:25.704 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:58:25.783 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:25 smithi042 ceph-mon[98894]: pgmap v317: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:25.783 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:25 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/54053978' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-12T09:58:25.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:25.812 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2023-12-12T09:58:25.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:25 smithi173 ceph-mon[103308]: pgmap v317: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:25.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:25 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/54053978' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-12T09:58:25.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:25.870 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2023-12-12T09:58:25.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:25.927 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:58:25.928 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:58:25.928 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-12T09:58:25.928 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-12T09:58:25.928 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:58:25.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:25 smithi111 ceph-mon[102565]: pgmap v317: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2023-12-12T09:58:25.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:25 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/54053978' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-12T09:58:26.026 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-12T09:58:26.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:26.027 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:58:26.027 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:58:26.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-12T09:58:26.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-12T09:58:26.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-12T09:58:26.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-12T09:58:26.027 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-12T09:58:26.027 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:58:26.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:26.224 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec bash -c ' 2023-12-12T09:58:26.224 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2023-12-12T09:58:26.224 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2023-12-12T09:58:26.224 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2023-12-12T09:58:26.224 DEBUG:teuthology.orchestra.run.smithi042:> ') 2023-12-12T09:58:26.344 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:58:26.344 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-12T09:58:26.367 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-12T09:58:26.367 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-12T09:58:26.367 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-12T09:58:26.425 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-12-12T09:58:26.425 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-12-12T09:58:26.425 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/8a17282b-a5de-4add-a6b2-6f3029871f8b 2023-12-12T09:58:26.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:26.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/8a17282b-a5de-4add-a6b2-6f3029871f8b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2023-12-12T09:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2023-12-12T09:58:26.514 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2023-12-12T09:58:26.731 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2023-12-12T09:58:26.731 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2023-12-12T09:58:26.731 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: resolved to: "172.21.15.42:3300,172.21.15.111:3300,172.21.15.173:3300" 2023-12-12T09:58:26.731 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: trying mount with new device syntax: 0@968ccaec-98d3-11ee-95a3-87774f69a715.cephfs=/volumes/_nogroup/sv_1/8a17282b-a5de-4add-a6b2-6f3029871f8b 2023-12-12T09:58:26.731 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.42:3300/172.21.15.111:3300/172.21.15.173:3300" will pass to kernel 2023-12-12T09:58:26.731 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: v2 mount support check returned 2 2023-12-12T09:58:26.732 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: kernel does not support v2 syntax 2023-12-12T09:58:26.732 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.42:3300,172.21.15.111:3300,172.21.15.173:3300:/volumes/_nogroup/sv_1/8a17282b-a5de-4add-a6b2-6f3029871f8b 2023-12-12T09:58:26.732 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mds_namespace=cephfs,fsid=968ccaec-98d3-11ee-95a3-87774f69a715" will pass to kernel 2023-12-12T09:58:26.734 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-12T09:58:26.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:26.734 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-12T09:58:26.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:26.766 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:58:26.868 INFO:teuthology.orchestra.run.smithi042.stdout:client15504 2023-12-12T09:58:26.868 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:58:26.868 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T09:58:26.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:26.887 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:58:26.989 INFO:teuthology.orchestra.run.smithi042.stdout:client15504 2023-12-12T09:58:26.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:26.989 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-12T09:58:27.042 INFO:teuthology.orchestra.run.smithi042.stdout:client15504 2023-12-12T09:58:27.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T09:58:27.043 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/968ccaec-98d3-11ee-95a3-87774f69a715.client15504/status') 2023-12-12T09:58:27.103 INFO:teuthology.orchestra.run.smithi042.stdout:instance: client.15504 (3)192.168.0.1:0/3179490943 2023-12-12T09:58:27.103 INFO:teuthology.orchestra.run.smithi042.stdout:blocklisted: false 2023-12-12T09:58:27.103 INFO:teuthology.orchestra.run.smithi042.stderr:0+1 records in 2023-12-12T09:58:27.103 INFO:teuthology.orchestra.run.smithi042.stderr:0+1 records out 2023-12-12T09:58:27.103 INFO:teuthology.orchestra.run.smithi042.stderr:70 bytes copied, 2.594e-05 s, 2.7 MB/s 2023-12-12T09:58:27.105 INFO:teuthology.run_tasks:Running task check-counter... 2023-12-12T09:58:27.117 INFO:teuthology.run_tasks:Running task exec... 2023-12-12T09:58:27.127 INFO:teuthology.task.exec:Executing custom commands... 2023-12-12T09:58:27.127 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T09:58:27.128 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-12-12T09:58:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.250+0000 7f98ee5f1700 1 -- 172.21.15.42:0/629128401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8101fb0 msgr2=0x7f98e8108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.250+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/629128401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8101fb0 0x7f98e8108860 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f98e000b0d0 tx=0x7f98e000b3e0 comp rx=0 tx=0).stop 2023-12-12T09:58:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.250+0000 7f98ee5f1700 1 -- 172.21.15.42:0/629128401 shutdown_connections 2023-12-12T09:58:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.250+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/629128401 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8101fb0 0x7f98e8108860 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.250+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/629128401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e8100420 0x7f98e8100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:27.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.250+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/629128401 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8101610 0x7f98e8101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.250+0000 7f98ee5f1700 1 -- 172.21.15.42:0/629128401 >> 172.21.15.42:0/629128401 conn(0x7f98e80fb990 msgr2=0x7f98e80fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.251+0000 7f98ee5f1700 1 -- 172.21.15.42:0/629128401 shutdown_connections 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.251+0000 7f98ee5f1700 1 -- 172.21.15.42:0/629128401 wait complete. 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.251+0000 7f98ee5f1700 1 Processor -- start 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.251+0000 7f98ee5f1700 1 -- start start 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98ee5f1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8100420 0x7f98e81964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98ee5f1700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8101610 0x7f98e81969e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98ee5f1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e8101fb0 0x7f98e819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98ee5f1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e819dfd0 con 0x7f98e8100420 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98ee5f1700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e819e110 con 0x7f98e8101610 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98ee5f1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e819e2a0 con 0x7f98e8101fb0 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8100420 0x7f98e81964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8100420 0x7f98e81964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35368/0 (socket says 172.21.15.42:35368) 2023-12-12T09:58:27.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e7fff700 1 -- 172.21.15.42:0/824731960 learned_addr learned my addr 172.21.15.42:0/824731960 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e77fe700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8101610 0x7f98e81969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98ecb8e700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e8101fb0 0x7f98e819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e7fff700 1 -- 172.21.15.42:0/824731960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e8101fb0 msgr2=0x7f98e819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e7fff700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e8101fb0 0x7f98e819da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e7fff700 1 -- 172.21.15.42:0/824731960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8101610 msgr2=0x7f98e81969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e7fff700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8101610 0x7f98e81969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e7fff700 1 -- 172.21.15.42:0/824731960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e0009d00 con 0x7f98e8100420 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.252+0000 7f98e7fff700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8100420 0x7f98e81964a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f98d800d980 tx=0x7f98d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.253+0000 7f98e57fa700 1 -- 172.21.15.42:0/824731960 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8004020 con 0x7f98e8100420 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.253+0000 7f98e57fa700 1 -- 172.21.15.42:0/824731960 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98d8009730 con 0x7f98e8100420 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.253+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e819e580 con 0x7f98e8100420 2023-12-12T09:58:27.258 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.253+0000 7f98e57fa700 1 -- 172.21.15.42:0/824731960 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8003a80 con 0x7f98e8100420 2023-12-12T09:58:27.259 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.253+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e819eae0 con 0x7f98e8100420 2023-12-12T09:58:27.259 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.254+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e8107820 con 0x7f98e8100420 2023-12-12T09:58:27.259 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.255+0000 7f98e57fa700 1 -- 172.21.15.42:0/824731960 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) v1 ==== 136057+0+0 (secure 0 0 0) 0x7f98d80098a0 con 0x7f98e8100420 2023-12-12T09:58:27.259 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.255+0000 7f98e57fa700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98d00a1590 0x7f98d00a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:27.260 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.256+0000 7f98e57fa700 1 -- 172.21.15.42:0/824731960 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f98d80cce90 con 0x7f98e8100420 2023-12-12T09:58:27.260 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.256+0000 7f98e77fe700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98d00a1590 0x7f98d00a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:27.260 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.256+0000 7f98e77fe700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98d00a1590 0x7f98d00a3a50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f98dc007be0 tx=0x7f98dc0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:27.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.260+0000 7f98e57fa700 1 -- 172.21.15.42:0/824731960 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f98d8096020 con 0x7f98e8100420 2023-12-12T09:58:27.529 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:27.527+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f98e810bf10 con 0x7f98e8100420 2023-12-12T09:58:27.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:27 smithi173 ceph-mon[103308]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:27.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:27 smithi042 ceph-mon[98894]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:27.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:27 smithi111 ceph-mon[102565]: pgmap v318: 129 pgs: 129 active+clean; 642 KiB data, 330 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2023-12-12T09:58:28.518 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.515+0000 7f98e57fa700 1 -- 172.21.15.42:0/824731960 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v18) v1 ==== 92+0+0 (secure 0 0 0) 0x7f98d8099640 con 0x7f98e8100420 2023-12-12T09:58:28.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.520+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98d00a1590 msgr2=0x7f98d00a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:28.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.520+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98d00a1590 0x7f98d00a3a50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f98dc007be0 tx=0x7f98dc0073d0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.520+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8100420 msgr2=0x7f98e81964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:28.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.520+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8100420 0x7f98e81964a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f98d800d980 tx=0x7f98d800dc90 comp rx=0 tx=0).stop 2023-12-12T09:58:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.524+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 shutdown_connections 2023-12-12T09:58:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.524+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f98d00a1590 0x7f98d00a3a50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.524+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98e8101fb0 0x7f98e819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.524+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f98e8100420 0x7f98e81964a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.524+0000 7f98ee5f1700 1 --2- 172.21.15.42:0/824731960 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f98e8101610 0x7f98e81969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.524+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 >> 172.21.15.42:0/824731960 conn(0x7f98e80fb990 msgr2=0x7f98e8104960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:28.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.526+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 shutdown_connections 2023-12-12T09:58:28.528 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.526+0000 7f98ee5f1700 1 -- 172.21.15.42:0/824731960 wait complete. 2023-12-12T09:58:28.544 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-12-12T09:58:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.694+0000 7f7b4e01f700 1 -- 172.21.15.42:0/3863928808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48107200 msgr2=0x7f7b48109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.694+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/3863928808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48107200 0x7f7b48109670 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000b0a0 tx=0x7f7b4000b3b0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.694+0000 7f7b4e01f700 1 -- 172.21.15.42:0/3863928808 shutdown_connections 2023-12-12T09:58:28.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.694+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/3863928808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b48107200 0x7f7b48109670 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.694+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/3863928808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101fa0 0x7f7b48104390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.694+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/3863928808 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b481048d0 0x7f7b48106cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.694+0000 7f7b4e01f700 1 -- 172.21.15.42:0/3863928808 >> 172.21.15.42:0/3863928808 conn(0x7f7b480fb990 msgr2=0x7f7b480fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:28.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.694+0000 7f7b4e01f700 1 -- 172.21.15.42:0/3863928808 shutdown_connections 2023-12-12T09:58:28.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.694+0000 7f7b4e01f700 1 -- 172.21.15.42:0/3863928808 wait complete. 2023-12-12T09:58:28.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b4e01f700 1 Processor -- start 2023-12-12T09:58:28.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b4e01f700 1 -- start start 2023-12-12T09:58:28.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b4e01f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101fa0 0x7f7b481975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:28.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b4e01f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b481048d0 0x7f7b48197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:28.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b4e01f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48107200 0x7f7b4819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:28.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b4e01f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4819f100 con 0x7f7b481048d0 2023-12-12T09:58:28.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b4e01f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4819f240 con 0x7f7b48107200 2023-12-12T09:58:28.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b4e01f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4819f3d0 con 0x7f7b48101fa0 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b46ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b481048d0 0x7f7b48197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b46ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b481048d0 0x7f7b48197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35412/0 (socket says 172.21.15.42:35412) 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b46ffd700 1 -- 172.21.15.42:0/1375881432 learned_addr learned my addr 172.21.15.42:0/1375881432 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b477fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101fa0 0x7f7b481975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.695+0000 7f7b47fff700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48107200 0x7f7b4819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b477fe700 1 -- 172.21.15.42:0/1375881432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48107200 msgr2=0x7f7b4819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b477fe700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48107200 0x7f7b4819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b477fe700 1 -- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b481048d0 msgr2=0x7f7b48197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b477fe700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b481048d0 0x7f7b48197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b477fe700 1 -- 172.21.15.42:0/1375881432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c009520 con 0x7f7b48101fa0 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b46ffd700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b481048d0 0x7f7b48197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b47fff700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48107200 0x7f7b4819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b477fe700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101fa0 0x7f7b481975d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800e9f0 tx=0x7f7b3800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:28.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b44ff9700 1 -- 172.21.15.42:0/1375881432 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b38004220 con 0x7f7b48101fa0 2023-12-12T09:58:28.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b44ff9700 1 -- 172.21.15.42:0/1375881432 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b38004cf0 con 0x7f7b48101fa0 2023-12-12T09:58:28.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b40009d00 con 0x7f7b48101fa0 2023-12-12T09:58:28.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b44ff9700 1 -- 172.21.15.42:0/1375881432 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800ce70 con 0x7f7b48101fa0 2023-12-12T09:58:28.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.696+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4819f9e0 con 0x7f7b48101fa0 2023-12-12T09:58:28.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.698+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b480fd510 con 0x7f7b48101fa0 2023-12-12T09:58:28.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.698+0000 7f7b44ff9700 1 -- 172.21.15.42:0/1375881432 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 18) v1 ==== 136023+0+0 (secure 0 0 0) 0x7f7b38004750 con 0x7f7b48101fa0 2023-12-12T09:58:28.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.699+0000 7f7b44ff9700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b300a14d0 0x7f7b300a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:28.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.699+0000 7f7b46ffd700 1 -- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b300a14d0 msgr2=0x7f7b300a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:28.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.699+0000 7f7b44ff9700 1 -- 172.21.15.42:0/1375881432 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7b38014070 con 0x7f7b48101fa0 2023-12-12T09:58:28.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.699+0000 7f7b46ffd700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b300a14d0 0x7f7b300a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:58:28.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.703+0000 7f7b44ff9700 1 -- 172.21.15.42:0/1375881432 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7b38096270 con 0x7f7b48101fa0 2023-12-12T09:58:28.774 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/824731960' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-12T09:58:28.775 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-12-12T09:58:28.775 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-12-12T09:58:28.775 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-12-12T09:58:28.775 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: ignoring --setuser ceph since I am not root 2023-12-12T09:58:28.775 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: ignoring --setgroup ceph since I am not root 2023-12-12T09:58:28.775 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-mgr[104044]: -- 172.21.15.111:0/986463782 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55864e6fb520 con 0x55864e6d0c00 2023-12-12T09:58:28.775 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:28.692+0000 7fb008e95200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:58:28.775 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:28.772+0000 7fb008e95200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:58:28.779 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/824731960' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-12T09:58:28.780 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-12-12T09:58:28.780 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-12-12T09:58:28.780 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-12-12T09:58:28.780 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: ignoring --setuser ceph since I am not root 2023-12-12T09:58:28.780 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: ignoring --setgroup ceph since I am not root 2023-12-12T09:58:28.780 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:28.696+0000 7f3a31670200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:58:28.780 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:28.777+0000 7f3a31670200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:58:28.853 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:28 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: ignoring --setuser ceph since I am not root 2023-12-12T09:58:28.854 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:28 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: ignoring --setgroup ceph since I am not root 2023-12-12T09:58:28.854 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:28 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:28.751+0000 7f9497bc5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-12T09:58:28.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:28 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/824731960' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-12T09:58:28.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-12-12T09:58:28.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]: dispatch 2023-12-12T09:58:28.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:28 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2023-12-12T09:58:28.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.900+0000 7f7b46ffd700 1 -- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b300a14d0 msgr2=0x7f7b300a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:28.902 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.900+0000 7f7b46ffd700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b300a14d0 0x7f7b300a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-12T09:58:28.934 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.932+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f7b48068450 con 0x7f7b48101fa0 2023-12-12T09:58:28.941 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.939+0000 7f7b44ff9700 1 -- 172.21.15.42:0/1375881432 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7b38004e60 con 0x7f7b48101fa0 2023-12-12T09:58:28.943 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b300a14d0 msgr2=0x7f7b300a3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b300a14d0 0x7f7b300a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101fa0 msgr2=0x7f7b481975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101fa0 0x7f7b481975d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800e9f0 tx=0x7f7b3800ed00 comp rx=0 tx=0).stop 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 shutdown_connections 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7b300a14d0 0x7f7b300a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b48107200 0x7f7b4819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b48101fa0 0x7f7b481975d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 --2- 172.21.15.42:0/1375881432 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b481048d0 0x7f7b48197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 >> 172.21.15.42:0/1375881432 conn(0x7f7b480fb990 msgr2=0x7f7b48102900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 shutdown_connections 2023-12-12T09:58:28.944 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:28.941+0000 7f7b4e01f700 1 -- 172.21.15.42:0/1375881432 wait complete. 2023-12-12T09:58:28.957 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-12-12T09:58:29.073 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:28.866+0000 7f3a31670200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:58:29.073 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:28.961+0000 7f3a31670200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:58:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 -- 172.21.15.42:0/3466110587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01019e0 msgr2=0x7f01c0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3466110587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01019e0 0x7f01c0101e00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f01b4009990 tx=0x7f01b4009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 -- 172.21.15.42:0/3466110587 shutdown_connections 2023-12-12T09:58:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3466110587 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01c01010b0 0x7f01c0108970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3466110587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01019e0 0x7f01c0101e00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3466110587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0100710 0x7f01c0100b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 -- 172.21.15.42:0/3466110587 >> 172.21.15.42:0/3466110587 conn(0x7f01c00fb160 msgr2=0x7f01c00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 -- 172.21.15.42:0/3466110587 shutdown_connections 2023-12-12T09:58:29.088 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 -- 172.21.15.42:0/3466110587 wait complete. 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 Processor -- start 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.084+0000 7f01c71ee700 1 -- start start 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01c71ee700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0100710 0x7f01c01974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01c71ee700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01010b0 0x7f01c0197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01c71ee700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01c01019e0 0x7f01c019ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01c71ee700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c019eff0 con 0x7f01c01010b0 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01c71ee700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c019f130 con 0x7f01c01019e0 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01c71ee700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c019f2c0 con 0x7f01c0100710 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01010b0 0x7f01c0197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01010b0 0x7f01c0197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35460/0 (socket says 172.21.15.42:35460) 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01c4f8a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0100710 0x7f01c01974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bffff700 1 -- 172.21.15.42:0/3932750069 learned_addr learned my addr 172.21.15.42:0/3932750069 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01c578b700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01c01019e0 0x7f01c019ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:29.089 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bffff700 1 -- 172.21.15.42:0/3932750069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0100710 msgr2=0x7f01c01974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bffff700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0100710 0x7f01c01974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bffff700 1 -- 172.21.15.42:0/3932750069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01c01019e0 msgr2=0x7f01c019ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bffff700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01c01019e0 0x7f01c019ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bffff700 1 -- 172.21.15.42:0/3932750069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b8009c40 con 0x7f01c01010b0 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bffff700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01010b0 0x7f01c0197a00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00e9a0 tx=0x7f01ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bdffb700 1 -- 172.21.15.42:0/3932750069 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac009790 con 0x7f01c01010b0 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01bdffb700 1 -- 172.21.15.42:0/3932750069 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f01ac00ce80 con 0x7f01c01010b0 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.085+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b40095f0 con 0x7f01c01010b0 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.086+0000 7f01bdffb700 1 -- 172.21.15.42:0/3932750069 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac018680 con 0x7f01c01010b0 2023-12-12T09:58:29.090 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.086+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c019f920 con 0x7f01c01010b0 2023-12-12T09:58:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.087+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c00688a0 con 0x7f01c01010b0 2023-12-12T09:58:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.088+0000 7f01bdffb700 1 -- 172.21.15.42:0/3932750069 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136023+0+0 (secure 0 0 0) 0x7f01ac010430 con 0x7f01c01010b0 2023-12-12T09:58:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.088+0000 7f01bdffb700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01b00a14d0 0x7f01b00a3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.088+0000 7f01c4f8a700 1 -- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01b00a14d0 msgr2=0x7f01b00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.089+0000 7f01bdffb700 1 -- 172.21.15.42:0/3932750069 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f01ac014070 con 0x7f01c01010b0 2023-12-12T09:58:29.091 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.089+0000 7f01c4f8a700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01b00a14d0 0x7f01b00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:58:29.094 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.093+0000 7f01bdffb700 1 -- 172.21.15.42:0/3932750069 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f01ac0522f0 con 0x7f01c01010b0 2023-12-12T09:58:29.127 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:28.861+0000 7fb008e95200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:58:29.127 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:28 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:28.955+0000 7fb008e95200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:58:29.153 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:28 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:28.850+0000 7f9497bc5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-12T09:58:29.153 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:28 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:28.963+0000 7f9497bc5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-12T09:58:29.154 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:29.082+0000 7f9497bc5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-12T09:58:29.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.290+0000 7f01c4f8a700 1 -- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01b00a14d0 msgr2=0x7f01b00a3990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:29.292 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.290+0000 7f01c4f8a700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01b00a14d0 0x7f01b00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-12T09:58:29.302 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.300+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f01c019fc80 con 0x7f01c01010b0 2023-12-12T09:58:29.305 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.303+0000 7f01bdffb700 1 -- 172.21.15.42:0/3932750069 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f01ac094e60 con 0x7f01c01010b0 2023-12-12T09:58:29.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01b00a14d0 msgr2=0x7f01b00a3990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:58:29.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01b00a14d0 0x7f01b00a3990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01010b0 msgr2=0x7f01c0197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:29.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01010b0 0x7f01c0197a00 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00e9a0 tx=0x7f01ac00ecb0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 shutdown_connections 2023-12-12T09:58:29.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f01b00a14d0 0x7f01b00a3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f01c01019e0 0x7f01c019ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.308 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c0100710 0x7f01c01974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.309 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 --2- 172.21.15.42:0/3932750069 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01c01010b0 0x7f01c0197a00 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.309 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 >> 172.21.15.42:0/3932750069 conn(0x7f01c00fb160 msgr2=0x7f01c00ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:29.309 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 shutdown_connections 2023-12-12T09:58:29.309 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.306+0000 7f01c71ee700 1 -- 172.21.15.42:0/3932750069 wait complete. 2023-12-12T09:58:29.321 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.449+0000 7f80129d7700 1 -- 172.21.15.42:0/2205689474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f800c107260 msgr2=0x7f800c109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.449+0000 7f80129d7700 1 --2- 172.21.15.42:0/2205689474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f800c107260 0x7f800c109670 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f800400b0a0 tx=0x7f800400b3b0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.449+0000 7f80129d7700 1 -- 172.21.15.42:0/2205689474 shutdown_connections 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.449+0000 7f80129d7700 1 --2- 172.21.15.42:0/2205689474 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f800c102000 0x7f800c1043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.449+0000 7f80129d7700 1 --2- 172.21.15.42:0/2205689474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f800c107260 0x7f800c109670 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.449+0000 7f80129d7700 1 --2- 172.21.15.42:0/2205689474 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c104930 0x7f800c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.449+0000 7f80129d7700 1 -- 172.21.15.42:0/2205689474 >> 172.21.15.42:0/2205689474 conn(0x7f800c0fb990 msgr2=0x7f800c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.449+0000 7f80129d7700 1 -- 172.21.15.42:0/2205689474 shutdown_connections 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.449+0000 7f80129d7700 1 -- 172.21.15.42:0/2205689474 wait complete. 2023-12-12T09:58:29.454 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f80129d7700 1 Processor -- start 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f80129d7700 1 -- start start 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f80129d7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f800c102000 0x7f800c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f80129d7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c104930 0x7f800c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f80129d7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f800c107260 0x7f800c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f80129d7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1a1320 con 0x7f800c102000 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f80129d7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1a1460 con 0x7f800c107260 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f80129d7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1a15f0 con 0x7f800c104930 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f800bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f800c102000 0x7f800c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f800bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f800c102000 0x7f800c1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35500/0 (socket says 172.21.15.42:35500) 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f800bfff700 1 -- 172.21.15.42:0/279884620 learned_addr learned my addr 172.21.15.42:0/279884620 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f8010f74700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f800c107260 0x7f800c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.450+0000 7f800b7fe700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c104930 0x7f800c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f800bfff700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c104930 msgr2=0x7f800c199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:29.455 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f800bfff700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f800c104930 0x7f800c199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f800bfff700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f800c107260 msgr2=0x7f800c1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f800bfff700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f800c107260 0x7f800c1a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f800bfff700 1 -- 172.21.15.42:0/279884620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8004009d00 con 0x7f800c102000 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f8010f74700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f800c107260 0x7f800c1a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f800bfff700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f800c102000 0x7f800c1997f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00e9a0 tx=0x7f7ffc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc009790 con 0x7f800c102000 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ffc010430 con 0x7f800c102000 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c1a18d0 con 0x7f800c102000 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc018680 con 0x7f800c102000 2023-12-12T09:58:29.456 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.451+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c1a1e30 con 0x7f800c102000 2023-12-12T09:58:29.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.452+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c108de0 con 0x7f800c102000 2023-12-12T09:58:29.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.453+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) v1 ==== 136023+0+0 (secure 0 0 0) 0x7f7ffc0105a0 con 0x7f800c102000 2023-12-12T09:58:29.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.454+0000 7f80097fa700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 0x7f7ff40a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:29.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.454+0000 7f800b7fe700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 msgr2=0x7f7ff40a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:29.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.454+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f7ffc014070 con 0x7f800c102000 2023-12-12T09:58:29.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.455+0000 7f800b7fe700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 0x7f7ff40a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-12T09:58:29.460 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.458+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7ffc0967b0 con 0x7f800c102000 2023-12-12T09:58:29.497 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:29.125+0000 7fb008e95200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:58:29.506 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:29.131+0000 7f3a31670200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:58:29.653 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:29.299+0000 7f9497bc5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-12T09:58:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/824731960' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-12T09:58:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-mon[98894]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-12-12T09:58:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-mon[98894]: from='client.? ' entity='client.admin' 2023-12-12T09:58:29.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-12-12T09:58:29.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-12-12T09:58:29.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-mon[98894]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-12-12T09:58:29.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-mon[98894]: osdmap e90: 12 total, 12 up, 12 in 2023-12-12T09:58:29.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3932750069' entity='client.admin' 2023-12-12T09:58:29.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.656+0000 7f800b7fe700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 msgr2=0x7f7ff40a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:29.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.656+0000 7f800b7fe700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 0x7f7ff40a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-12T09:58:29.693 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:29.691+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 --> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f800c02d380 con 0x7f7ff40a1550 2023-12-12T09:58:29.815 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:29.498+0000 7fb008e95200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:58:29.815 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:29.630+0000 7fb008e95200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:58:29.816 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:29.718+0000 7fb008e95200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:58:29.816 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/824731960' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-12T09:58:29.816 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-mon[102565]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-12-12T09:58:29.816 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-mon[102565]: from='client.? ' entity='client.admin' 2023-12-12T09:58:29.816 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-12-12T09:58:29.816 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-12-12T09:58:29.816 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-mon[102565]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-12-12T09:58:29.816 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-mon[102565]: osdmap e90: 12 total, 12 up, 12 in 2023-12-12T09:58:29.816 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3932750069' entity='client.admin' 2023-12-12T09:58:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/824731960' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-12T09:58:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-mon[103308]: mgrmap e18: x(active, since 8m), standbys: z, y 2023-12-12T09:58:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-mon[103308]: from='client.? ' entity='client.admin' 2023-12-12T09:58:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-12-12T09:58:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2023-12-12T09:58:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-mon[103308]: from='mgr.14152 172.21.15.42:0/3920316820' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2023-12-12T09:58:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-mon[103308]: osdmap e90: 12 total, 12 up, 12 in 2023-12-12T09:58:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3932750069' entity='client.admin' 2023-12-12T09:58:29.823 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:29.504+0000 7f3a31670200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:58:29.824 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:29.637+0000 7f3a31670200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:58:29.824 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:29.726+0000 7f3a31670200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:58:29.824 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:29.820+0000 7f3a31670200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:58:30.057 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:29.790+0000 7f9497bc5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-12T09:58:30.057 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:29 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:29.945+0000 7f9497bc5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-12T09:58:30.058 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:30.057+0000 7f800b7fe700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 msgr2=0x7f7ff40a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:30.058 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:30.057+0000 7f800b7fe700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 0x7f7ff40a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-12T09:58:30.403 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:30.055+0000 7f9497bc5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-12T09:58:30.403 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:30.171+0000 7f9497bc5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:58:30.497 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:29.813+0000 7fb008e95200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-12T09:58:30.497 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:30 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:30.232+0000 7fb008e95200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:58:30.573 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:30 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:30.241+0000 7f3a31670200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:58:30.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:30.858+0000 7f800b7fe700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 msgr2=0x7f7ff40a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:30.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:30.858+0000 7f800b7fe700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 0x7f7ff40a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-12T09:58:31.153 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:30.694+0000 7f9497bc5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-12T09:58:31.247 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:30 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:30.894+0000 7fb008e95200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:58:31.247 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:30 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:30.976+0000 7fb008e95200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:58:31.294 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:30 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:30.904+0000 7f3a31670200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:58:31.294 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:30 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:30.986+0000 7f3a31670200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:58:31.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:31 smithi173 ceph-mon[103308]: osdmap e91: 12 total, 12 up, 12 in 2023-12-12T09:58:31.589 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:31.486+0000 7f9497bc5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-12T09:58:31.589 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:31 smithi042 ceph-mon[98894]: osdmap e91: 12 total, 12 up, 12 in 2023-12-12T09:58:31.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:31 smithi111 ceph-mon[102565]: osdmap e91: 12 total, 12 up, 12 in 2023-12-12T09:58:31.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:31.587+0000 7f9497bc5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-12T09:58:32.020 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:31 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:31.753+0000 7fb008e95200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:58:32.020 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:31 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:31.844+0000 7fb008e95200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:58:32.026 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:31 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:31.761+0000 7f3a31670200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:58:32.026 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:31 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:31.852+0000 7f3a31670200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:58:32.323 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:32.024+0000 7f3a31670200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:58:32.436 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:32.017+0000 7fb008e95200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:58:32.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:32.459+0000 7f800b7fe700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 msgr2=0x7f7ff40a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:32.461 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:32.459+0000 7f800b7fe700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 0x7f7ff40a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-12T09:58:32.747 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:32.434+0000 7fb008e95200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:58:32.747 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:32.522+0000 7fb008e95200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:58:32.747 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:32.672+0000 7fb008e95200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:58:32.823 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:32.447+0000 7f3a31670200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:58:32.823 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:32.535+0000 7f3a31670200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:58:32.823 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:32.681+0000 7f3a31670200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:58:32.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:32 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:32.574+0000 7f9497bc5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:58:32.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:32 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:32.687+0000 7f9497bc5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-12T09:58:33.247 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:32.936+0000 7fb008e95200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:58:33.323 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:32.943+0000 7f3a31670200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:58:33.403 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:32 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:32.915+0000 7f9497bc5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-12T09:58:33.718 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:33.421+0000 7f9497bc5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-12T09:58:33.718 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:33.529+0000 7f9497bc5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-12T09:58:33.962 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:33.633+0000 7fb008e95200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:58:33.962 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:33.763+0000 7fb008e95200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:58:33.974 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:33.647+0000 7f3a31670200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:58:33.974 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:33.778+0000 7f3a31670200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:58:34.058 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:33.715+0000 7f9497bc5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-12T09:58:34.247 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:33.957+0000 7fb008e95200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:58:34.247 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:34 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:34.069+0000 7fb008e95200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:58:34.303 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:33.972+0000 7f3a31670200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:58:34.303 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:34 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:34.084+0000 7f3a31670200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:58:34.403 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:34.055+0000 7f9497bc5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-12T09:58:34.573 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:34 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:34.300+0000 7f3a31670200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:58:34.747 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:34 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:34.287+0000 7fb008e95200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:58:35.153 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:34.893+0000 7f9497bc5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-12T09:58:35.153 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:35.056+0000 7f9497bc5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-12T09:58:35.224 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:35.222+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 19) v1 ==== 136023+0+0 (secure 0 0 0) 0x7f7ffc010880 con 0x7f800c102000 2023-12-12T09:58:35.497 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y[104020]: 2023-12-12T09:58:35.145+0000 7fb008e95200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: Standby manager daemon y restarted 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: Standby manager daemon y started 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: Standby manager daemon z restarted 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: Standby manager daemon z started 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:58:35.498 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-12T09:58:35.499 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:35 smithi111 ceph-mon[102565]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:58:35.573 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z[105377]: 2023-12-12T09:58:35.162+0000 7f3a31670200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:58:35.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: Standby manager daemon y restarted 2023-12-12T09:58:35.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: Standby manager daemon y started 2023-12-12T09:58:35.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-12T09:58:35.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:58:35.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-12T09:58:35.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:58:35.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: Standby manager daemon z restarted 2023-12-12T09:58:35.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: Standby manager daemon z started 2023-12-12T09:58:35.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-12T09:58:35.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:58:35.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-12T09:58:35.574 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:35 smithi173 ceph-mon[103308]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:58:35.642 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:35.320+0000 7f9497bc5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-12T09:58:35.642 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:35.460+0000 7f9497bc5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-12T09:58:35.642 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: Standby manager daemon y restarted 2023-12-12T09:58:35.642 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: Standby manager daemon y started 2023-12-12T09:58:35.642 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-12T09:58:35.642 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:58:35.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-12T09:58:35.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.111:0/2606894148' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:58:35.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: Standby manager daemon z restarted 2023-12-12T09:58:35.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: Standby manager daemon z started 2023-12-12T09:58:35.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-12T09:58:35.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-12T09:58:35.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-12T09:58:35.643 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-mon[98894]: from='mgr.? 172.21.15.173:0/4185605391' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-12T09:58:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:35.662+0000 7f800b7fe700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 msgr2=0x7f7ff40a3a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/2637927159 2023-12-12T09:58:35.664 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:35.662+0000 7f800b7fe700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 0x7f7ff40a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-12-12T09:58:35.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:35.744+0000 7f9497bc5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-12T09:58:36.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:36 smithi111 ceph-mon[102565]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-12-12T09:58:36.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:36 smithi173 ceph-mon[103308]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-12-12T09:58:36.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:36 smithi042 ceph-mon[98894]: mgrmap e19: x(active, since 8m), standbys: z, y 2023-12-12T09:58:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:36.835+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 20) v1 ==== 135691+0+0 (secure 0 0 0) 0x7f7ffc096b80 con 0x7f800c102000 2023-12-12T09:58:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:36.835+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 msgr2=0x7f7ff40a3a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T09:58:36.837 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:36.835+0000 7f80097fa700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/2637927159,v1:172.21.15.42:6801/2637927159] conn(0x7f7ff40a1550 0x7f7ff40a3a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:37.100 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:36.811+0000 7f9497bc5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.039+0000 7f93f9d27700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.039+0000 7f93f9d27700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.039+0000 7f93f9d27700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.039+0000 7f93f9d27700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.039+0000 7f93f9d27700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.040+0000 7f93f6d21700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.040+0000 7f93f6d21700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.040+0000 7f93f6d21700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.040+0000 7f93f6d21700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.101 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:58:37.040+0000 7f93f6d21700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:58:37.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: Active manager daemon x restarted 2023-12-12T09:58:37.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: Activating manager daemon x 2023-12-12T09:58:37.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: osdmap e92: 12 total, 12 up, 12 in 2023-12-12T09:58:37.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: mgrmap e20: x(active, starting, since 0.0124222s), standbys: z, y 2023-12-12T09:58:37.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-12T09:58:37.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:58:37.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: Manager daemon x is now available 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-12T09:58:37.251 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:37 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-12T09:58:37.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: Active manager daemon x restarted 2023-12-12T09:58:37.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: Activating manager daemon x 2023-12-12T09:58:37.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: osdmap e92: 12 total, 12 up, 12 in 2023-12-12T09:58:37.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: mgrmap e20: x(active, starting, since 0.0124222s), standbys: z, y 2023-12-12T09:58:37.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:58:37.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-12T09:58:37.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-12T09:58:37.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-12T09:58:37.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-12T09:58:37.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: Manager daemon x is now available 2023-12-12T09:58:37.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:58:37.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:58:37.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-12T09:58:37.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:37 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-12T09:58:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: Active manager daemon x restarted 2023-12-12T09:58:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: Activating manager daemon x 2023-12-12T09:58:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: osdmap e92: 12 total, 12 up, 12 in 2023-12-12T09:58:37.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: mgrmap e20: x(active, starting, since 0.0124222s), standbys: z, y 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-12T09:58:37.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-12T09:58:37.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-12T09:58:37.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-12T09:58:37.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-12T09:58:37.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-12T09:58:37.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-12T09:58:37.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: Manager daemon x is now available 2023-12-12T09:58:37.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:58:37.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:58:37.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-12T09:58:37.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:37 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-12T09:58:37.845 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.842+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mon.0 v2:172.21.15.42:3300/0 9 ==== mgrmap(e 21) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f7ffc0cac20 con 0x7f800c102000 2023-12-12T09:58:37.845 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.843+0000 7f80097fa700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7ff4027bc0 0x7f7ff4027fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:37.845 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.843+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 --> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7f800c02d380 con 0x7f7ff4027bc0 2023-12-12T09:58:37.849 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.847+0000 7f800b7fe700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7ff4027bc0 0x7f7ff4027fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:37.851 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.849+0000 7f800b7fe700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7ff4027bc0 0x7f7ff4027fa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8000005950 tx=0x7f800000b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:37.914 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.912+0000 7f80097fa700 1 -- 172.21.15.42:0/279884620 <== mgr.15528 v2:172.21.15.42:6800/1776845983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f800c02d380 con 0x7f7ff4027bc0 2023-12-12T09:58:37.914 INFO:teuthology.orchestra.run.smithi042.stdout:Schedule set for path / 2023-12-12T09:58:37.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.914+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7ff4027bc0 msgr2=0x7f7ff4027fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:37.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.914+0000 7f80129d7700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7ff4027bc0 0x7f7ff4027fa0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8000005950 tx=0x7f800000b410 comp rx=0 tx=0).stop 2023-12-12T09:58:37.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.914+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f800c102000 msgr2=0x7f800c1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:37.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.914+0000 7f80129d7700 1 --2- 172.21.15.42:0/279884620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f800c102000 0x7f800c1997f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00e9a0 tx=0x7f7ffc00ecb0 comp rx=0 tx=0).stop 2023-12-12T09:58:37.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.914+0000 7f8010f74700 1 -- 172.21.15.42:0/279884620 reap_dead start 2023-12-12T09:58:37.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.914+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 shutdown_connections 2023-12-12T09:58:37.916 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.914+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 >> 172.21.15.42:0/279884620 conn(0x7f800c0fb990 msgr2=0x7f800c101760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:37.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.914+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 shutdown_connections 2023-12-12T09:58:37.917 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:37.914+0000 7f80129d7700 1 -- 172.21.15.42:0/279884620 wait complete. 2023-12-12T09:58:37.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2023-12-12T09:58:38.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/3653447362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0102030 msgr2=0x7fd0a0104420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/3653447362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0102030 0x7fd0a0104420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd09c009990 tx=0x7fd09c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/3653447362 shutdown_connections 2023-12-12T09:58:38.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/3653447362 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a0107290 0x7fd0a01096a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/3653447362 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0102030 0x7fd0a0104420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/3653447362 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0a0104960 0x7fd0a0106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/3653447362 >> 172.21.15.42:0/3653447362 conn(0x7fd0a00fb9c0 msgr2=0x7fd0a00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:38.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/3653447362 shutdown_connections 2023-12-12T09:58:38.070 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/3653447362 wait complete. 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 Processor -- start 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 -- start start 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a0102030 0x7fd0a01996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0a0104960 0x7fd0a0199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.066+0000 7fd0a7e1d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0107290 0x7fd0a01a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a7e1d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a01a1220 con 0x7fd0a0107290 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a7e1d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a01a1360 con 0x7fd0a0104960 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a7e1d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a01a14f0 con 0x7fd0a0102030 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a63ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0107290 0x7fd0a01a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a63ba700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0107290 0x7fd0a01a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35768/0 (socket says 172.21.15.42:35768) 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a63ba700 1 -- 172.21.15.42:0/1616033767 learned_addr learned my addr 172.21.15.42:0/1616033767 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a5bb9700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a0102030 0x7fd0a01996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a53b8700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0a0104960 0x7fd0a0199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a63ba700 1 -- 172.21.15.42:0/1616033767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a0102030 msgr2=0x7fd0a01996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.071 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a63ba700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a0102030 0x7fd0a01996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a63ba700 1 -- 172.21.15.42:0/1616033767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0a0104960 msgr2=0x7fd0a0199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a63ba700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0a0104960 0x7fd0a0199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a63ba700 1 -- 172.21.15.42:0/1616033767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c0095f0 con 0x7fd0a0107290 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a53b8700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0a0104960 0x7fd0a0199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a63ba700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0107290 0x7fd0a01a0c50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd09800ada0 tx=0x7fd09800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd096ffd700 1 -- 172.21.15.42:0/1616033767 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09800f070 con 0x7fd0a0107290 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd096ffd700 1 -- 172.21.15.42:0/1616033767 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd098007b30 con 0x7fd0a0107290 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a01a17d0 con 0x7fd0a0107290 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd096ffd700 1 -- 172.21.15.42:0/1616033767 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0980053e0 con 0x7fd0a0107290 2023-12-12T09:58:38.072 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.067+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a01a1ce0 con 0x7fd0a0107290 2023-12-12T09:58:38.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.068+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a0108e30 con 0x7fd0a0107290 2023-12-12T09:58:38.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.070+0000 7fd096ffd700 1 -- 172.21.15.42:0/1616033767 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) v1 ==== 135818+0+0 (secure 0 0 0) 0x7fd09800c980 con 0x7fd0a0107290 2023-12-12T09:58:38.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.071+0000 7fd096ffd700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd08c0a1300 0x7fd08c0a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.071+0000 7fd096ffd700 1 -- 172.21.15.42:0/1616033767 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd0980cc820 con 0x7fd0a0107290 2023-12-12T09:58:38.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.071+0000 7fd0a5bb9700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd08c0a1300 0x7fd08c0a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.071+0000 7fd0a5bb9700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd08c0a1300 0x7fd08c0a37c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b5c0 tx=0x7fd09c005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:38.076 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.074+0000 7fd096ffd700 1 -- 172.21.15.42:0/1616033767 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd098095d90 con 0x7fd0a0107290 2023-12-12T09:58:38.317 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.315+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 --> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0a002d380 con 0x7fd08c0a1300 2023-12-12T09:58:38.368 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.366+0000 7fd096ffd700 1 -- 172.21.15.42:0/1616033767 <== mgr.15528 v2:172.21.15.42:6800/1776845983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fd0a002d380 con 0x7fd08c0a1300 2023-12-12T09:58:38.368 INFO:teuthology.orchestra.run.smithi042.stdout:Retention added to path / 2023-12-12T09:58:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd08c0a1300 msgr2=0x7fd08c0a37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd08c0a1300 0x7fd08c0a37c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00b5c0 tx=0x7fd09c005fd0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0107290 msgr2=0x7fd0a01a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0107290 0x7fd0a01a0c50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fd09800ada0 tx=0x7fd09800c330 comp rx=0 tx=0).stop 2023-12-12T09:58:38.371 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 shutdown_connections 2023-12-12T09:58:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd0a0107290 0x7fd0a01a0c50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a0102030 0x7fd0a01996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd08c0a1300 0x7fd08c0a37c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 --2- 172.21.15.42:0/1616033767 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd0a0104960 0x7fd0a0199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 >> 172.21.15.42:0/1616033767 conn(0x7fd0a00fb9c0 msgr2=0x7fd0a01017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 shutdown_connections 2023-12-12T09:58:38.372 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.368+0000 7fd0a7e1d700 1 -- 172.21.15.42:0/1616033767 wait complete. 2023-12-12T09:58:38.383 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-12T09:58:38.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:38 smithi111 ceph-mon[102565]: [12/Dec/2023:09:58:37] ENGINE Bus STARTING 2023-12-12T09:58:38.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:38 smithi111 ceph-mon[102565]: [12/Dec/2023:09:58:37] ENGINE Serving on https://172.21.15.42:7150 2023-12-12T09:58:38.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:38 smithi111 ceph-mon[102565]: mgrmap e21: x(active, since 1.02021s), standbys: z, y 2023-12-12T09:58:38.522 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.516+0000 7f6725afe700 1 -- 172.21.15.42:0/9187502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6720101ff0 msgr2=0x7f67201088a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.516+0000 7f6725afe700 1 --2- 172.21.15.42:0/9187502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6720101ff0 0x7f67201088a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f671800b0d0 tx=0x7f671800b3e0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.516+0000 7f6725afe700 1 -- 172.21.15.42:0/9187502 shutdown_connections 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.516+0000 7f6725afe700 1 --2- 172.21.15.42:0/9187502 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6720101ff0 0x7f67201088a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.516+0000 7f6725afe700 1 --2- 172.21.15.42:0/9187502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6720100460 0x7f6720100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.516+0000 7f6725afe700 1 --2- 172.21.15.42:0/9187502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6720101650 0x7f6720101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.516+0000 7f6725afe700 1 -- 172.21.15.42:0/9187502 >> 172.21.15.42:0/9187502 conn(0x7f67200fb9d0 msgr2=0x7f67200fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.516+0000 7f6725afe700 1 -- 172.21.15.42:0/9187502 shutdown_connections 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.516+0000 7f6725afe700 1 -- 172.21.15.42:0/9187502 wait complete. 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f6725afe700 1 Processor -- start 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f6725afe700 1 -- start start 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f6725afe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6720100460 0x7f672006a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.523 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f6725afe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6720101650 0x7f672019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f6725afe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6720101ff0 0x7f67201a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f6725afe700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67201a12b0 con 0x7f6720101650 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f6725afe700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67201a1420 con 0x7f6720101ff0 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f6725afe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67201a15e0 con 0x7f6720100460 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f671effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6720101650 0x7f672019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6720100460 0x7f672006a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6720100460 0x7f672006a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:50762/0 (socket says 172.21.15.42:50762) 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f671effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6720101650 0x7f672019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35780/0 (socket says 172.21.15.42:35780) 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f671f7fe700 1 -- 172.21.15.42:0/3550374996 learned_addr learned my addr 172.21.15.42:0/3550374996 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.517+0000 7f671ffff700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6720101ff0 0x7f67201a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671f7fe700 1 -- 172.21.15.42:0/3550374996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6720101ff0 msgr2=0x7f67201a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671f7fe700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6720101ff0 0x7f67201a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671f7fe700 1 -- 172.21.15.42:0/3550374996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6720101650 msgr2=0x7f672019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.524 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671f7fe700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6720101650 0x7f672019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671f7fe700 1 -- 172.21.15.42:0/3550374996 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710009520 con 0x7f6720100460 2023-12-12T09:58:38.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671ffff700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6720101ff0 0x7f67201a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:58:38.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671f7fe700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6720100460 0x7f672006a270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f671000ea50 tx=0x7f671000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:38.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671cff9700 1 -- 172.21.15.42:0/3550374996 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000cbd0 con 0x7f6720100460 2023-12-12T09:58:38.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6718009d00 con 0x7f6720100460 2023-12-12T09:58:38.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671cff9700 1 -- 172.21.15.42:0/3550374996 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f671000cd30 con 0x7f6720100460 2023-12-12T09:58:38.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f671cff9700 1 -- 172.21.15.42:0/3550374996 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710018680 con 0x7f6720100460 2023-12-12T09:58:38.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.518+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67201a1c90 con 0x7f6720100460 2023-12-12T09:58:38.525 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.519+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6720107890 con 0x7f6720100460 2023-12-12T09:58:38.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.521+0000 7f671cff9700 1 -- 172.21.15.42:0/3550374996 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 21) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f6710010430 con 0x7f6720100460 2023-12-12T09:58:38.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.522+0000 7f671cff9700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f67080a1280 0x7f67080a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.522+0000 7f671effd700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f67080a1280 0x7f67080a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.522+0000 7f671cff9700 1 -- 172.21.15.42:0/3550374996 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6710014070 con 0x7f6720100460 2023-12-12T09:58:38.526 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.522+0000 7f671effd700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f67080a1280 0x7f67080a3740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f672006a830 tx=0x7f6714009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:38.527 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.525+0000 7f671cff9700 1 -- 172.21.15.42:0/3550374996 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f6710096d50 con 0x7f6720100460 2023-12-12T09:58:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:38 smithi173 ceph-mon[103308]: [12/Dec/2023:09:58:37] ENGINE Bus STARTING 2023-12-12T09:58:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:38 smithi173 ceph-mon[103308]: [12/Dec/2023:09:58:37] ENGINE Serving on https://172.21.15.42:7150 2023-12-12T09:58:38.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:38 smithi173 ceph-mon[103308]: mgrmap e21: x(active, since 1.02021s), standbys: z, y 2023-12-12T09:58:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:38 smithi042 ceph-mon[98894]: [12/Dec/2023:09:58:37] ENGINE Bus STARTING 2023-12-12T09:58:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:38 smithi042 ceph-mon[98894]: [12/Dec/2023:09:58:37] ENGINE Serving on https://172.21.15.42:7150 2023-12-12T09:58:38.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:38 smithi042 ceph-mon[98894]: mgrmap e21: x(active, since 1.02021s), standbys: z, y 2023-12-12T09:58:38.765 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.763+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 --> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f6720061a10 con 0x7f67080a1280 2023-12-12T09:58:38.775 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.773+0000 7f671cff9700 1 -- 172.21.15.42:0/3550374996 <== mgr.15528 v2:172.21.15.42:6800/1776845983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f6720061a10 con 0x7f67080a1280 2023-12-12T09:58:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-12-12T00:00:00", "created": "2023-12-12T09:58:37", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-12-12T09:58:38.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f67080a1280 msgr2=0x7f67080a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f67080a1280 0x7f67080a3740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f672006a830 tx=0x7f6714009260 comp rx=0 tx=0).stop 2023-12-12T09:58:38.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6720100460 msgr2=0x7f672006a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6720100460 0x7f672006a270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f671000ea50 tx=0x7f671000ed60 comp rx=0 tx=0).stop 2023-12-12T09:58:38.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 shutdown_connections 2023-12-12T09:58:38.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f67080a1280 0x7f67080a3740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6720101ff0 0x7f67201a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6720100460 0x7f672006a270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.779 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 --2- 172.21.15.42:0/3550374996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6720101650 0x7f672019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 >> 172.21.15.42:0/3550374996 conn(0x7f67200fb9d0 msgr2=0x7f67201049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:38.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 shutdown_connections 2023-12-12T09:58:38.780 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.776+0000 7f6725afe700 1 -- 172.21.15.42:0/3550374996 wait complete. 2023-12-12T09:58:38.793 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-12T09:58:38.935 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.933+0000 7f4bf20f9700 1 -- 172.21.15.42:0/3087127191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 msgr2=0x7f4bec100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.935 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.933+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/3087127191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 0x7f4bec100c00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009990 tx=0x7f4be0009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.935 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.933+0000 7f4bf20f9700 1 -- 172.21.15.42:0/3087127191 shutdown_connections 2023-12-12T09:58:38.936 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.933+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/3087127191 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec101140 0x7f4bec1015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.936 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.933+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/3087127191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bec0ffe20 0x7f4bec100240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.936 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.933+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/3087127191 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 0x7f4bec100c00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.936 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.933+0000 7f4bf20f9700 1 -- 172.21.15.42:0/3087127191 >> 172.21.15.42:0/3087127191 conn(0x7f4bec0fb9c0 msgr2=0x7f4bec0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:38.936 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.933+0000 7f4bf20f9700 1 -- 172.21.15.42:0/3087127191 shutdown_connections 2023-12-12T09:58:38.936 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.933+0000 7f4bf20f9700 1 -- 172.21.15.42:0/3087127191 wait complete. 2023-12-12T09:58:38.938 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4bf20f9700 1 Processor -- start 2023-12-12T09:58:38.938 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4bf20f9700 1 -- start start 2023-12-12T09:58:38.938 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4bf20f9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0ffe20 0x7f4bec199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4bf20f9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 0x7f4bec199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4bf20f9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bec101140 0x7f4bec1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4bf20f9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec1a11a0 con 0x7f4bec100780 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4bf20f9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec1a12e0 con 0x7f4bec0ffe20 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4bf20f9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec1a1470 con 0x7f4bec101140 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4beaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 0x7f4bec199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4beaffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 0x7f4bec199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35796/0 (socket says 172.21.15.42:35796) 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.934+0000 7f4beaffd700 1 -- 172.21.15.42:0/1552454004 learned_addr learned my addr 172.21.15.42:0/1552454004 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4bebfff700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bec101140 0x7f4bec1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4beaffd700 1 -- 172.21.15.42:0/1552454004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bec101140 msgr2=0x7f4bec1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4beaffd700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bec101140 0x7f4bec1a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4beaffd700 1 -- 172.21.15.42:0/1552454004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0ffe20 msgr2=0x7f4bec199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4beaffd700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0ffe20 0x7f4bec199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:38.939 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4beaffd700 1 -- 172.21.15.42:0/1552454004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00095f0 con 0x7f4bec100780 2023-12-12T09:58:38.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4beaffd700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 0x7f4bec199bb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4be000b2e0 tx=0x7f4be00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:38.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4be8ff9700 1 -- 172.21.15.42:0/1552454004 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be001d070 con 0x7f4bec100780 2023-12-12T09:58:38.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4be8ff9700 1 -- 172.21.15.42:0/1552454004 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4be0022470 con 0x7f4bec100780 2023-12-12T09:58:38.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec1a16f0 con 0x7f4bec100780 2023-12-12T09:58:38.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4be8ff9700 1 -- 172.21.15.42:0/1552454004 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0017650 con 0x7f4bec100780 2023-12-12T09:58:38.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.935+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec1a1b10 con 0x7f4bec100780 2023-12-12T09:58:38.940 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.936+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec10ae90 con 0x7f4bec100780 2023-12-12T09:58:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.938+0000 7f4be8ff9700 1 -- 172.21.15.42:0/1552454004 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 21) v1 ==== 135818+0+0 (secure 0 0 0) 0x7f4be00225e0 con 0x7f4bec100780 2023-12-12T09:58:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.938+0000 7f4be8ff9700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4bd40a1280 0x7f4bd40a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.939+0000 7f4be8ff9700 1 -- 172.21.15.42:0/1552454004 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4be00ce5d0 con 0x7f4bec100780 2023-12-12T09:58:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.940+0000 7f4beb7fe700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4bd40a1280 0x7f4bd40a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:38.942 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.940+0000 7f4beb7fe700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4bd40a1280 0x7f4bd40a3740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005950 tx=0x7f4bdc0058e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:38.945 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:38.943+0000 7f4be8ff9700 1 -- 172.21.15.42:0/1552454004 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4be0097b40 con 0x7f4bec100780 2023-12-12T09:58:39.179 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.177+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 --> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4bec02d380 con 0x7f4bd40a1280 2023-12-12T09:58:39.186 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.184+0000 7f4be8ff9700 1 -- 172.21.15.42:0/1552454004 <== mgr.15528 v2:172.21.15.42:6800/1776845983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f4bec02d380 con 0x7f4bd40a1280 2023-12-12T09:58:39.186 INFO:teuthology.orchestra.run.smithi042.stdout:/ 1m 6m3h 2023-12-12T09:58:39.189 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4bd40a1280 msgr2=0x7f4bd40a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:39.189 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4bd40a1280 0x7f4bd40a3740 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc005950 tx=0x7f4bdc0058e0 comp rx=0 tx=0).stop 2023-12-12T09:58:39.189 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 msgr2=0x7f4bec199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:39.190 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 0x7f4bec199bb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4be000b2e0 tx=0x7f4be00057f0 comp rx=0 tx=0).stop 2023-12-12T09:58:39.190 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 shutdown_connections 2023-12-12T09:58:39.190 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4bd40a1280 0x7f4bd40a3740 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:39.190 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bec0ffe20 0x7f4bec199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:39.190 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4bec101140 0x7f4bec1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:39.190 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 --2- 172.21.15.42:0/1552454004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4bec100780 0x7f4bec199bb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:39.190 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.186+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 >> 172.21.15.42:0/1552454004 conn(0x7f4bec0fb9c0 msgr2=0x7f4bec0fd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:39.191 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.188+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 shutdown_connections 2023-12-12T09:58:39.192 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T09:58:39.188+0000 7f4bf20f9700 1 -- 172.21.15.42:0/1552454004 wait complete. 2023-12-12T09:58:39.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-12-12T09:58:39.236 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-12-12T09:58:39.248 INFO:teuthology.run_tasks:Running task background_exec... 2023-12-12T09:58:39.262 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T09:58:39.262 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-12-12T09:58:39.278 INFO:teuthology.run_tasks:Running task workunit... 2023-12-12T09:58:39.292 INFO:tasks.workunit:Pulling workunits from ref f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:58:39.292 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-12-12T09:58:39.292 INFO:tasks.workunit:timeout=3h 2023-12-12T09:58:39.292 INFO:tasks.workunit:cleanup=True 2023-12-12T09:58:39.293 DEBUG:teuthology.orchestra.run.smithi042:> stat -- /home/ubuntu/cephtest/mnt.0 2023-12-12T09:58:39.312 INFO:teuthology.orchestra.run.smithi042.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-12-12T09:58:39.312 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-12-12T09:58:39.312 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-12-12T09:58:39.312 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-12T09:58:39.312 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:cephfs_t:s0 2023-12-12T09:58:39.313 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2023-12-12 09:57:06.524597985 +0000 2023-12-12T09:58:39.313 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2023-12-12 09:57:06.524597985 +0000 2023-12-12T09:58:39.313 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2023-12-12 09:58:26.760647570 +0000 2023-12-12T09:58:39.313 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2023-12-12 09:57:06.524597985 +0000 2023-12-12T09:58:39.313 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-12-12T09:58:39.313 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-12-12T09:58:39.411 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 2023-12-12T09:58:39.429 INFO:tasks.workunit.client.0.smithi042.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-12-12T09:58:39.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:39 smithi173 ceph-mon[103308]: [12/Dec/2023:09:58:37] ENGINE Serving on http://172.21.15.42:8765 2023-12-12T09:58:39.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:39 smithi173 ceph-mon[103308]: [12/Dec/2023:09:58:37] ENGINE Bus STARTED 2023-12-12T09:58:39.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:39 smithi173 ceph-mon[103308]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:39.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:39 smithi173 ceph-mon[103308]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:58:39.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:39 smithi173 ceph-mon[103308]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:39 smithi042 ceph-mon[98894]: [12/Dec/2023:09:58:37] ENGINE Serving on http://172.21.15.42:8765 2023-12-12T09:58:39.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:39 smithi042 ceph-mon[98894]: [12/Dec/2023:09:58:37] ENGINE Bus STARTED 2023-12-12T09:58:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:39 smithi042 ceph-mon[98894]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:39 smithi042 ceph-mon[98894]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:58:39.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:39 smithi042 ceph-mon[98894]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:39 smithi111 ceph-mon[102565]: [12/Dec/2023:09:58:37] ENGINE Serving on http://172.21.15.42:8765 2023-12-12T09:58:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:39 smithi111 ceph-mon[102565]: [12/Dec/2023:09:58:37] ENGINE Bus STARTED 2023-12-12T09:58:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:39 smithi111 ceph-mon[102565]: from='client.15546 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:39 smithi111 ceph-mon[102565]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:58:39.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:39 smithi111 ceph-mon[102565]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:40 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:58:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-12-12T09:58:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:40 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: from='client.25270 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:40.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:58:40.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: from='client.15597 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T09:58:40.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: mgrmap e22: x(active, since 3s), standbys: z, y 2023-12-12T09:58:40.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:40.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:40 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi111 to 196.2M 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Unable to set osd_memory_target on smithi111 to 205768192: error parsing value: Value '205768192' is below minimum 939524096 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi173 to 196.2M 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Unable to set osd_memory_target on smithi173 to 205771776: error parsing value: Value '205771776' is below minimum 939524096 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi042 to 197.2M 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Unable to set osd_memory_target on smithi042 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Updating smithi042:/etc/ceph/ceph.conf 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Updating smithi111:/etc/ceph/ceph.conf 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Updating smithi173:/etc/ceph/ceph.conf 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: mgrmap e23: x(active, since 4s), standbys: z, y 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:58:42.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:41 smithi111 ceph-mon[102565]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:58:42.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:58:42.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi111 to 196.2M 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Unable to set osd_memory_target on smithi111 to 205768192: error parsing value: Value '205768192' is below minimum 939524096 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi173 to 196.2M 2023-12-12T09:58:42.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Unable to set osd_memory_target on smithi173 to 205771776: error parsing value: Value '205771776' is below minimum 939524096 2023-12-12T09:58:42.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi042 to 197.2M 2023-12-12T09:58:42.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Unable to set osd_memory_target on smithi042 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-12-12T09:58:42.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:58:42.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:58:42.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Updating smithi042:/etc/ceph/ceph.conf 2023-12-12T09:58:42.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Updating smithi111:/etc/ceph/ceph.conf 2023-12-12T09:58:42.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Updating smithi173:/etc/ceph/ceph.conf 2023-12-12T09:58:42.328 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: mgrmap e23: x(active, since 4s), standbys: z, y 2023-12-12T09:58:42.328 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:58:42.328 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:58:42.328 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:41 smithi173 ceph-mon[103308]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi111 to 196.2M 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Unable to set osd_memory_target on smithi111 to 205768192: error parsing value: Value '205768192' is below minimum 939524096 2023-12-12T09:58:42.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi173 to 196.2M 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Unable to set osd_memory_target on smithi173 to 205771776: error parsing value: Value '205771776' is below minimum 939524096 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-12T09:58:42.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi042 to 197.2M 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Unable to set osd_memory_target on smithi042 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Updating smithi042:/etc/ceph/ceph.conf 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Updating smithi111:/etc/ceph/ceph.conf 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Updating smithi173:/etc/ceph/ceph.conf 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: mgrmap e23: x(active, since 4s), standbys: z, y 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:58:42.406 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:41 smithi042 ceph-mon[98894]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.conf 2023-12-12T09:58:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:58:43.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:58:43.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:58:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:58:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:58:43.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:58:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: Updating smithi111:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:58:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: Updating smithi173:/etc/ceph/ceph.client.admin.keyring 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: Updating smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: Updating smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: Updating smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/config/ceph.client.admin.keyring 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:58:43.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:58:44.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:44 smithi111 ceph-mon[102565]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 410 B/s wr, 1 op/s 2023-12-12T09:58:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:44 smithi173 ceph-mon[103308]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 410 B/s wr, 1 op/s 2023-12-12T09:58:44.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:44 smithi042 ceph-mon[98894]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 332 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 410 B/s wr, 1 op/s 2023-12-12T09:58:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:46 smithi111 ceph-mon[102565]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 71 op/s 2023-12-12T09:58:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:46 smithi173 ceph-mon[103308]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 71 op/s 2023-12-12T09:58:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:46 smithi042 ceph-mon[98894]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 71 op/s 2023-12-12T09:58:48.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:48 smithi111 ceph-mon[102565]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-12-12T09:58:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:48 smithi173 ceph-mon[103308]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-12-12T09:58:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:48 smithi042 ceph-mon[98894]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2023-12-12T09:58:50.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:50 smithi111 ceph-mon[102565]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-12-12T09:58:50.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:50 smithi173 ceph-mon[103308]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-12-12T09:58:50.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:50 smithi042 ceph-mon[98894]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2023-12-12T09:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.449+0000 7fc726dd0700 1 -- 172.21.15.42:0/2594183019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 msgr2=0x7fc72010d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.449+0000 7fc726dd0700 1 --2- 172.21.15.42:0/2594183019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 0x7fc72010d810 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc70c009990 tx=0x7fc70c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.449+0000 7fc726dd0700 1 -- 172.21.15.42:0/2594183019 shutdown_connections 2023-12-12T09:58:52.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.449+0000 7fc726dd0700 1 --2- 172.21.15.42:0/2594183019 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc720073980 0x7fc720073d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.449+0000 7fc726dd0700 1 --2- 172.21.15.42:0/2594183019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc72010dd50 0x7fc720110140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.449+0000 7fc726dd0700 1 --2- 172.21.15.42:0/2594183019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 0x7fc72010d810 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.449+0000 7fc726dd0700 1 -- 172.21.15.42:0/2594183019 >> 172.21.15.42:0/2594183019 conn(0x7fc7200fcfa0 msgr2=0x7fc7200ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.449+0000 7fc726dd0700 1 -- 172.21.15.42:0/2594183019 shutdown_connections 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.449+0000 7fc726dd0700 1 -- 172.21.15.42:0/2594183019 wait complete. 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.450+0000 7fc726dd0700 1 Processor -- start 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.450+0000 7fc726dd0700 1 -- start start 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc726dd0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc720073980 0x7fc72019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc726dd0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 0x7fc72019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc726dd0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc72010dd50 0x7fc7201999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc726dd0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72019a040 con 0x7fc7200742a0 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc726dd0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72019a1b0 con 0x7fc720073980 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc726dd0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc72019a370 con 0x7fc72010dd50 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 0x7fc72019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 0x7fc72019fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35830/0 (socket says 172.21.15.42:35830) 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71ffff700 1 -- 172.21.15.42:0/1272857509 learned_addr learned my addr 172.21.15.42:0/1272857509 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc724b6c700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc720073980 0x7fc72019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc72536d700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc72010dd50 0x7fc7201999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71ffff700 1 -- 172.21.15.42:0/1272857509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc72010dd50 msgr2=0x7fc7201999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71ffff700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc72010dd50 0x7fc7201999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71ffff700 1 -- 172.21.15.42:0/1272857509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc720073980 msgr2=0x7fc72019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71ffff700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc720073980 0x7fc72019f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71ffff700 1 -- 172.21.15.42:0/1272857509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc70c0095f0 con 0x7fc7200742a0 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71ffff700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 0x7fc72019fdf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000c00 tx=0x7fc70c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:52.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71dffb700 1 -- 172.21.15.42:0/1272857509 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c01d070 con 0x7fc7200742a0 2023-12-12T09:58:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc726dd0700 1 -- 172.21.15.42:0/1272857509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc72019a5f0 con 0x7fc7200742a0 2023-12-12T09:58:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71dffb700 1 -- 172.21.15.42:0/1272857509 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc70c022470 con 0x7fc7200742a0 2023-12-12T09:58:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.451+0000 7fc71dffb700 1 -- 172.21.15.42:0/1272857509 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c017650 con 0x7fc7200742a0 2023-12-12T09:58:52.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.452+0000 7fc726dd0700 1 -- 172.21.15.42:0/1272857509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7201a68a0 con 0x7fc7200742a0 2023-12-12T09:58:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.454+0000 7fc71dffb700 1 -- 172.21.15.42:0/1272857509 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 23) v1 ==== 136182+0+0 (secure 0 0 0) 0x7fc70c0225e0 con 0x7fc7200742a0 2023-12-12T09:58:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.454+0000 7fc71dffb700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fc7100a1680 0x7fc7100a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.455+0000 7fc71dffb700 1 -- 172.21.15.42:0/1272857509 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc70c0ce790 con 0x7fc7200742a0 2023-12-12T09:58:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.455+0000 7fc724b6c700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fc7100a1680 0x7fc7100a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.455+0000 7fc724b6c700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fc7100a1680 0x7fc7100a3b40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc714009c60 tx=0x7fc714009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc732fc9b80 1 Processor -- start 2023-12-12T09:58:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc732fc9b80 1 -- start start 2023-12-12T09:58:52.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29ae660 0x5609e29a8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29a8ac0 0x5609e29a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e2903af0 con 0x5609e29add40 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e2901410 con 0x5609e29a8ac0 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e28fcd00 con 0x5609e29ae660 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc724b6c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc724b6c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35836/0 (socket says 172.21.15.42:35836) 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc724b6c700 1 -- 172.21.15.42:0/3457152391 learned_addr learned my addr 172.21.15.42:0/3457152391 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.456+0000 7fc72536d700 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29a8ac0 0x5609e29a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc71ffff700 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29ae660 0x5609e29a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc724b6c700 1 -- 172.21.15.42:0/3457152391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29ae660 msgr2=0x5609e29a8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc724b6c700 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29ae660 0x5609e29a8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc724b6c700 1 -- 172.21.15.42:0/3457152391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29a8ac0 msgr2=0x5609e29a8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc724b6c700 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29a8ac0 0x5609e29a8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc724b6c700 1 -- 172.21.15.42:0/3457152391 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e27795d0 con 0x5609e29add40 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc72536d700 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29a8ac0 0x5609e29a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc724b6c700 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29ae120 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc71400e4e0 tx=0x7fc714014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99943020b979359e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc70affd700 1 -- 172.21.15.42:0/3457152391 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400ee00 con 0x5609e29add40 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc70affd700 1 -- 172.21.15.42:0/3457152391 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc714021070 con 0x5609e29add40 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.457+0000 7fc70affd700 1 -- 172.21.15.42:0/3457152391 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71401d7b0 con 0x5609e29add40 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 -- 172.21.15.42:0/3457152391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 msgr2=0x5609e29ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29ae120 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fc71400e4e0 tx=0x7fc714014000 comp rx=0 tx=0).stop 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 -- 172.21.15.42:0/3457152391 shutdown_connections 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29a8ac0 0x5609e29a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29ae120 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/3457152391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29ae660 0x5609e29a8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 -- 172.21.15.42:0/3457152391 >> 172.21.15.42:0/3457152391 conn(0x5609e28d7780 msgr2=0x5609e29a2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 -- 172.21.15.42:0/3457152391 shutdown_connections 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 -- 172.21.15.42:0/3457152391 wait complete. 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 -- start start 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.458+0000 7fc732fc9b80 1 Processor -- start 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29a8ac0 0x5609e29b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 0x5609e29b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e28b6500 con 0x5609e29add40 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e28b61a0 con 0x5609e29ae660 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e277b8c0 con 0x5609e29a8ac0 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc732fc9b80 10 client.0 _ll_register_callbacks cb 0x5609e27e1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:52.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29b1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35844/0 (socket says 172.21.15.42:35844) 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc71ffff700 1 -- 172.21.15.42:0/2028952561 learned_addr learned my addr 172.21.15.42:0/2028952561 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc724b6c700 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29a8ac0 0x5609e29b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc72536d700 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 0x5609e29b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc71ffff700 1 -- 172.21.15.42:0/2028952561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29a8ac0 msgr2=0x5609e29b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc71ffff700 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29a8ac0 0x5609e29b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc71ffff700 1 -- 172.21.15.42:0/2028952561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 msgr2=0x5609e29b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc71ffff700 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 0x5609e29b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.459+0000 7fc71ffff700 1 -- 172.21.15.42:0/2028952561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc718011230 con 0x5609e29add40 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc71ffff700 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29b1ef0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc70c09b2d0 tx=0x7fc70c022bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6e2ed5a3d4e1f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc6fffff700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc6fffff700 1 -- 172.21.15.42:0/2028952561 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc70c0ce630 con 0x5609e29add40 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc6fffff700 1 -- 172.21.15.42:0/2028952561 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc70c0b2020 con 0x5609e29add40 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2028952561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609e299b730 con 0x5609e29add40 2023-12-12T09:58:52.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc6fffff700 1 -- 172.21.15.42:0/2028952561 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc70c09ba40 con 0x5609e29add40 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc732fc9b80 10 client.15615 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2028952561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc714017070 con 0x5609e29add40 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc6fffff700 1 -- 172.21.15.42:0/2028952561 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609e299b730 con 0x5609e29add40 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc6fffff700 1 -- 172.21.15.42:0/2028952561 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc70c0b9370 con 0x5609e29add40 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc732fc9b80 10 client.15615 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc732fc9b80 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc732fc9b80 20 client.15615 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5609e29c1dd0 0x5609e29c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc732fc9b80 4 client.15615 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.460+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2028952561 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609e2977ee0 con 0x5609e29c1dd0 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.461+0000 7fc724b6c700 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5609e29c1dd0 0x5609e29c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.461+0000 7fc724b6c700 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5609e29c1dd0 0x5609e29c41f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4055778058a08a server_cookie=e3028d49ed0d2ebc in_seq=0 out_seq=0 2023-12-12T09:58:52.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.461+0000 7fc6fffff700 10 client.15615 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc6fffff700 1 -- 172.21.15.42:0/2028952561 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5609e2977ee0 con 0x5609e29c1dd0 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc6fffff700 10 client.15615 handle_command_reply: tid=0 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 client.15615 shutdown 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2028952561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 msgr2=0x5609e29b1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29b1ef0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc70c09b2d0 tx=0x7fc70c022bc0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2028952561 shutdown_connections 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5609e29c1dd0 0x5609e29c41f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 0x5609e29b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc6fffff700 0 client.15615 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29a8ac0 0x5609e29b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2028952561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29add40 0x5609e29b1ef0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2028952561 >> 172.21.15.42:0/2028952561 conn(0x5609e28d7780 msgr2=0x5609e29a7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2028952561 shutdown_connections 2023-12-12T09:58:52.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2028952561 wait complete. 2023-12-12T09:58:52.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.462+0000 7fc732fc9b80 20 client.15615 trim_cache size 0 max 16384 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.477+0000 7fc732fc9b80 1 Processor -- start 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc732fc9b80 1 -- start start 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29add40 0x5609e29a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 0x5609e29b9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e2977ee0 con 0x5609e29a8ac0 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e29a4780 con 0x5609e29ae660 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e29b2fd0 con 0x5609e29add40 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc724b6c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc724b6c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35850/0 (socket says 172.21.15.42:35850) 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc724b6c700 1 -- 172.21.15.42:0/2708372593 learned_addr learned my addr 172.21.15.42:0/2708372593 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc72536d700 1 --2- 172.21.15.42:0/2708372593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 0x5609e29b9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc71ffff700 1 --2- 172.21.15.42:0/2708372593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29add40 0x5609e29a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc724b6c700 1 -- 172.21.15.42:0/2708372593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29add40 msgr2=0x5609e29a8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc724b6c700 1 --2- 172.21.15.42:0/2708372593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29add40 0x5609e29a8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc724b6c700 1 -- 172.21.15.42:0/2708372593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 msgr2=0x5609e29b9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc724b6c700 1 --2- 172.21.15.42:0/2708372593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 0x5609e29b9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc724b6c700 1 -- 172.21.15.42:0/2708372593 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e2725d80 con 0x5609e29a8ac0 2023-12-12T09:58:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.478+0000 7fc724b6c700 1 --2- 172.21.15.42:0/2708372593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29c41c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc714005920 tx=0x7fc71400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f002e374db85f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc70b7fe700 1 -- 172.21.15.42:0/2708372593 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714005040 con 0x5609e29a8ac0 2023-12-12T09:58:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc70b7fe700 1 -- 172.21.15.42:0/2708372593 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc714005660 con 0x5609e29a8ac0 2023-12-12T09:58:52.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc70b7fe700 1 -- 172.21.15.42:0/2708372593 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714033910 con 0x5609e29a8ac0 2023-12-12T09:58:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2708372593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 msgr2=0x5609e29c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2708372593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29c41c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc714005920 tx=0x7fc71400ec00 comp rx=0 tx=0).stop 2023-12-12T09:58:52.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2708372593 shutdown_connections 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2708372593 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29ae660 0x5609e29b9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2708372593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29c41c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/2708372593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29add40 0x5609e29a8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2708372593 >> 172.21.15.42:0/2708372593 conn(0x5609e28d7780 msgr2=0x5609e29b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2708372593 shutdown_connections 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.479+0000 7fc732fc9b80 1 -- 172.21.15.42:0/2708372593 wait complete. 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc732fc9b80 1 -- start start 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc732fc9b80 1 Processor -- start 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29b94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29add40 0x5609e29b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc732fc9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29ae660 0x5609e29b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e29b7670 con 0x5609e29a8ac0 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e29b77e0 con 0x5609e29add40 2023-12-12T09:58:52.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc732fc9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e2a17300 con 0x5609e29ae660 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc732fc9b80 10 client.0 _ll_register_callbacks cb 0x5609e27e1a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc724b6c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29b94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc724b6c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29b94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35856/0 (socket says 172.21.15.42:35856) 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc724b6c700 1 -- 172.21.15.42:0/958430661 learned_addr learned my addr 172.21.15.42:0/958430661 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc724b6c700 1 -- 172.21.15.42:0/958430661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29ae660 msgr2=0x5609e29b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc71ffff700 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29add40 0x5609e29b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc724b6c700 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29ae660 0x5609e29b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc724b6c700 1 -- 172.21.15.42:0/958430661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29add40 msgr2=0x5609e29b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc724b6c700 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29add40 0x5609e29b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc724b6c700 1 -- 172.21.15.42:0/958430661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71402a040 con 0x5609e29a8ac0 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.480+0000 7fc71ffff700 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29add40 0x5609e29b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc724b6c700 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29b94c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc714005950 tx=0x7fc714009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82d20b554cc25ab server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc6fffff700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc6fffff700 1 -- 172.21.15.42:0/958430661 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7140337d0 con 0x5609e29a8ac0 2023-12-12T09:58:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc6fffff700 1 -- 172.21.15.42:0/958430661 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc714033180 con 0x5609e29a8ac0 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc732fc9b80 1 -- 172.21.15.42:0/958430661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc714021070 con 0x5609e29a8ac0 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc6fffff700 1 -- 172.21.15.42:0/958430661 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc714033930 con 0x5609e29a8ac0 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc732fc9b80 10 client.15621 fetch_fsmap learned FSMap version 27 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc732fc9b80 1 -- 172.21.15.42:0/958430661 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5609e270ec00 con 0x5609e29a8ac0 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc6fffff700 1 -- 172.21.15.42:0/958430661 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc714045080 con 0x5609e29a8ac0 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc6fffff700 1 -- 172.21.15.42:0/958430661 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fc71403a070 con 0x5609e29a8ac0 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc732fc9b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc732fc9b80 10 client.15621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc732fc9b80 20 client.15621 populate_metadata read hostname 'smithi042' 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5609e2a18870 0x5609e2a1ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc732fc9b80 4 client.15621 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.481+0000 7fc732fc9b80 1 -- 172.21.15.42:0/958430661 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5609e28c2f70 con 0x5609e2a18870 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.482+0000 7fc71ffff700 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5609e2a18870 0x5609e2a1ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.482+0000 7fc6fffff700 10 client.15621 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:58:52.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.482+0000 7fc71ffff700 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5609e2a18870 0x5609e2a1ac90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6847fef2fc47be74 server_cookie=5b5fc29c6a45bb10 in_seq=0 out_seq=0 2023-12-12T09:58:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc6fffff700 1 -- 172.21.15.42:0/958430661 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5609e28c2f70 con 0x5609e2a18870 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc6fffff700 10 client.15621 handle_command_reply: tid=0 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 client.15621 shutdown 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 -- 172.21.15.42:0/958430661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 msgr2=0x5609e29b94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29b94c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc714005950 tx=0x7fc714009480 comp rx=0 tx=0).stop 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 -- 172.21.15.42:0/958430661 shutdown_connections 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5609e29ae660 0x5609e29b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5609e29a8ac0 0x5609e29b94c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5609e2a18870 0x5609e2a1ac90 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 --2- 172.21.15.42:0/958430661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5609e29add40 0x5609e29b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc6fffff700 0 client.15621 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 -- 172.21.15.42:0/958430661 >> 172.21.15.42:0/958430661 conn(0x5609e28d7780 msgr2=0x5609e299d240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 -- 172.21.15.42:0/958430661 shutdown_connections 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 1 -- 172.21.15.42:0/958430661 wait complete. 2023-12-12T09:58:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.496+0000 7fc732fc9b80 20 client.15621 trim_cache size 0 max 16384 2023-12-12T09:58:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T09:58:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T09:58:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T09:58:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 -- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fc7100a1680 msgr2=0x7fc7100a3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fc7100a1680 0x7fc7100a3b40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc714009c60 tx=0x7fc714009260 comp rx=0 tx=0).stop 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 -- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 msgr2=0x7fc72019fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 0x7fc72019fdf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc70c000c00 tx=0x7fc70c004730 comp rx=0 tx=0).stop 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 -- 172.21.15.42:0/1272857509 shutdown_connections 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fc7100a1680 0x7fc7100a3b40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc720073980 0x7fc72019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc72010dd50 0x7fc7201999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 --2- 172.21.15.42:0/1272857509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc7200742a0 0x7fc72019fdf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 -- 172.21.15.42:0/1272857509 >> 172.21.15.42:0/1272857509 conn(0x7fc7200fcfa0 msgr2=0x7fc7200fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 -- 172.21.15.42:0/1272857509 shutdown_connections 2023-12-12T09:58:52.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:58:52.497+0000 7fc726dd0700 1 -- 172.21.15.42:0/1272857509 wait complete. 2023-12-12T09:58:52.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:52 smithi111 ceph-mon[102565]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-12-12T09:58:52.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:52 smithi173 ceph-mon[103308]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-12-12T09:58:52.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:52 smithi042 ceph-mon[98894]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2023-12-12T09:58:54.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:54 smithi111 ceph-mon[102565]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-12-12T09:58:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:54 smithi173 ceph-mon[103308]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-12-12T09:58:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:54 smithi042 ceph-mon[98894]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 46 op/s 2023-12-12T09:58:56.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:56 smithi111 ceph-mon[102565]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 54 op/s 2023-12-12T09:58:56.822 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:56 smithi173 ceph-mon[103308]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 54 op/s 2023-12-12T09:58:56.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:56 smithi042 ceph-mon[98894]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 78 KiB/s wr, 54 op/s 2023-12-12T09:58:58.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:58:58 smithi111 ceph-mon[102565]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-12T09:58:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:58:58 smithi173 ceph-mon[103308]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-12T09:58:58.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:58:58 smithi042 ceph-mon[98894]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-12T09:59:00.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:00 smithi042 ceph-mon[98894]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2023-12-12T09:59:00.654 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:59:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:59:00.378+0000 7f93d8865700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:59:00.654 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:59:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:59:00.378+0000 7f93d8865700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:59:00.654 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:59:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:59:00.378+0000 7f93d8865700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:59:00.655 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:59:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:59:00.378+0000 7f93d8865700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:59:00.655 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 09:59:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T09:59:00.378+0000 7f93d8865700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-12T09:59:00.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:00 smithi111 ceph-mon[102565]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2023-12-12T09:59:00.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:00 smithi173 ceph-mon[103308]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.7 KiB/s wr, 17 op/s 2023-12-12T09:59:02.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:02 smithi111 ceph-mon[102565]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2023-12-12T09:59:02.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:02 smithi111 ceph-mon[102565]: mgrmap e24: x(active, since 24s), standbys: z, y 2023-12-12T09:59:02.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:02 smithi173 ceph-mon[103308]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2023-12-12T09:59:02.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:02 smithi173 ceph-mon[103308]: mgrmap e24: x(active, since 24s), standbys: z, y 2023-12-12T09:59:02.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:02 smithi042 ceph-mon[98894]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2023-12-12T09:59:02.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:02 smithi042 ceph-mon[98894]: mgrmap e24: x(active, since 24s), standbys: z, y 2023-12-12T09:59:04.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:04 smithi111 ceph-mon[102565]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 KiB/s wr, 17 op/s 2023-12-12T09:59:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:04 smithi173 ceph-mon[103308]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 KiB/s wr, 17 op/s 2023-12-12T09:59:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:04 smithi042 ceph-mon[98894]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 14 KiB/s wr, 17 op/s 2023-12-12T09:59:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 -- 172.21.15.42:0/959038444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c1003d0 msgr2=0x7f4e5c100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 --2- 172.21.15.42:0/959038444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c1003d0 0x7f4e5c100850 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009990 tx=0x7f4e4c009ca0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 -- 172.21.15.42:0/959038444 shutdown_connections 2023-12-12T09:59:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 --2- 172.21.15.42:0/959038444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e5c106480 0x7f4e5c106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 --2- 172.21.15.42:0/959038444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5c100d90 0x7f4e5c10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 --2- 172.21.15.42:0/959038444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c1003d0 0x7f4e5c100850 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 -- 172.21.15.42:0/959038444 >> 172.21.15.42:0/959038444 conn(0x7f4e5c077570 msgr2=0x7f4e5c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 -- 172.21.15.42:0/959038444 shutdown_connections 2023-12-12T09:59:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 -- 172.21.15.42:0/959038444 wait complete. 2023-12-12T09:59:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 Processor -- start 2023-12-12T09:59:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.620+0000 7f4e63acb700 1 -- start start 2023-12-12T09:59:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e63acb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5c1003d0 0x7f4e5c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e63acb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c100d90 0x7f4e5c19db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e63acb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e5c106480 0x7f4e5c197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e63acb700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c197db0 con 0x7f4e5c100d90 2023-12-12T09:59:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e63acb700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c197f20 con 0x7f4e5c106480 2023-12-12T09:59:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e63acb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c1980e0 con 0x7f4e5c1003d0 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61066700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c100d90 0x7f4e5c19db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61066700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c100d90 0x7f4e5c19db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35890/0 (socket says 172.21.15.42:35890) 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61066700 1 -- 172.21.15.42:0/1402680803 learned_addr learned my addr 172.21.15.42:0/1402680803 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61867700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5c1003d0 0x7f4e5c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e62068700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e5c106480 0x7f4e5c197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61066700 1 -- 172.21.15.42:0/1402680803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5c1003d0 msgr2=0x7f4e5c19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61066700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5c1003d0 0x7f4e5c19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61066700 1 -- 172.21.15.42:0/1402680803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e5c106480 msgr2=0x7f4e5c197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61066700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e5c106480 0x7f4e5c197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61066700 1 -- 172.21.15.42:0/1402680803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c0095f0 con 0x7f4e5c100d90 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e61066700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c100d90 0x7f4e5c19db10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005910 tx=0x7f4e4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.621+0000 7f4e52ffd700 1 -- 172.21.15.42:0/1402680803 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c01d070 con 0x7f4e5c100d90 2023-12-12T09:59:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.622+0000 7f4e63acb700 1 -- 172.21.15.42:0/1402680803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c198360 con 0x7f4e5c100d90 2023-12-12T09:59:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.622+0000 7f4e52ffd700 1 -- 172.21.15.42:0/1402680803 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c022470 con 0x7f4e5c100d90 2023-12-12T09:59:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.622+0000 7f4e52ffd700 1 -- 172.21.15.42:0/1402680803 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c017650 con 0x7f4e5c100d90 2023-12-12T09:59:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.622+0000 7f4e63acb700 1 -- 172.21.15.42:0/1402680803 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c1a45c0 con 0x7f4e5c100d90 2023-12-12T09:59:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.624+0000 7f4e52ffd700 1 -- 172.21.15.42:0/1402680803 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136239+0+0 (secure 0 0 0) 0x7f4e4c0225e0 con 0x7f4e5c100d90 2023-12-12T09:59:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.624+0000 7f4e52ffd700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e480a1710 0x7f4e480a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.624+0000 7f4e52ffd700 1 -- 172.21.15.42:0/1402680803 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4e4c0ce920 con 0x7f4e5c100d90 2023-12-12T09:59:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.624+0000 7f4e61867700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e480a1710 0x7f4e480a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.625+0000 7f4e61867700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e480a1710 0x7f4e480a3bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009c60 tx=0x7f4e58009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.625+0000 7f4e6fcc4b80 1 Processor -- start 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.625+0000 7f4e6fcc4b80 1 -- start start 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f1d40 0x557fb65f2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f2660 0x557fb65ec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 0x557fb65ecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb6547af0 con 0x557fb65f2660 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb6545410 con 0x557fb65f1d40 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb6540d00 con 0x557fb65ecac0 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61066700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f2660 0x557fb65ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61066700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f2660 0x557fb65ec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35898/0 (socket says 172.21.15.42:35898) 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61066700 1 -- 172.21.15.42:0/226402818 learned_addr learned my addr 172.21.15.42:0/226402818 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:05.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61867700 1 --2- 172.21.15.42:0/226402818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f1d40 0x557fb65f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e62068700 1 --2- 172.21.15.42:0/226402818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 0x557fb65ecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61066700 1 -- 172.21.15.42:0/226402818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 msgr2=0x557fb65ecf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61066700 1 --2- 172.21.15.42:0/226402818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 0x557fb65ecf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61066700 1 -- 172.21.15.42:0/226402818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f1d40 msgr2=0x557fb65f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61066700 1 --2- 172.21.15.42:0/226402818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f1d40 0x557fb65f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61066700 1 -- 172.21.15.42:0/226402818 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fb63bd5d0 con 0x557fb65f2660 2023-12-12T09:59:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e61066700 1 --2- 172.21.15.42:0/226402818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f2660 0x557fb65ec580 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c000c00 tx=0x7f4e4c022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfb48736cb8d9e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:05.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.626+0000 7f4e47fff700 1 -- 172.21.15.42:0/226402818 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c09ae70 con 0x557fb65f2660 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e47fff700 1 -- 172.21.15.42:0/226402818 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c09bc10 con 0x557fb65f2660 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e47fff700 1 -- 172.21.15.42:0/226402818 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c09aa00 con 0x557fb65f2660 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/226402818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f2660 msgr2=0x557fb65ec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/226402818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f2660 0x557fb65ec580 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c000c00 tx=0x7f4e4c022f90 comp rx=0 tx=0).stop 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/226402818 shutdown_connections 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/226402818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f1d40 0x557fb65f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/226402818 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 0x557fb65ecf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/226402818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f2660 0x557fb65ec580 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/226402818 >> 172.21.15.42:0/226402818 conn(0x557fb651b780 msgr2=0x557fb65e6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/226402818 shutdown_connections 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.627+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/226402818 wait complete. 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.628+0000 7f4e6fcc4b80 1 -- start start 2023-12-12T09:59:05.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.628+0000 7f4e6fcc4b80 1 Processor -- start 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.628+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65ecac0 0x557fb65f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f1d40 0x557fb65f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 0x557fb65f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb64fa500 con 0x557fb65ecac0 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb64fa1a0 con 0x557fb65f2660 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb63bf8c0 con 0x557fb65f1d40 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61867700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65ecac0 0x557fb65f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e6fcc4b80 10 client.0 _ll_register_callbacks cb 0x557fb6425a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61867700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65ecac0 0x557fb65f85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35902/0 (socket says 172.21.15.42:35902) 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61867700 1 -- 172.21.15.42:0/945549018 learned_addr learned my addr 172.21.15.42:0/945549018 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61066700 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f1d40 0x557fb65f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e62068700 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 0x557fb65f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61867700 1 -- 172.21.15.42:0/945549018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f1d40 msgr2=0x557fb65f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61867700 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f1d40 0x557fb65f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61867700 1 -- 172.21.15.42:0/945549018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 msgr2=0x557fb65f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61867700 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 0x557fb65f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61867700 1 -- 172.21.15.42:0/945549018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c1a45c0 con 0x557fb65ecac0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e61867700 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65ecac0 0x557fb65f85f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4e58005910 tx=0x7f4e580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d306892ad8181061 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e457fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e457fa700 1 -- 172.21.15.42:0/945549018 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58022070 con 0x557fb65ecac0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e457fa700 1 -- 172.21.15.42:0/945549018 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4e5801d720 con 0x557fb65ecac0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.629+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/945549018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557fb65df730 con 0x557fb65ecac0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e457fa700 1 -- 172.21.15.42:0/945549018 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e5801d8e0 con 0x557fb65ecac0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e6fcc4b80 10 client.15639 fetch_fsmap learned FSMap version 27 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/945549018 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557fb6369d80 con 0x557fb65ecac0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e457fa700 1 -- 172.21.15.42:0/945549018 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557fb65df730 con 0x557fb65ecac0 2023-12-12T09:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e457fa700 1 -- 172.21.15.42:0/945549018 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4e58027b70 con 0x557fb65ecac0 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e6fcc4b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e6fcc4b80 10 client.15639 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e6fcc4b80 20 client.15639 populate_metadata read hostname 'smithi042' 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557fb6605d70 0x557fb6608190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e6fcc4b80 4 client.15639 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/945549018 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fb65bbee0 con 0x557fb6605d70 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.630+0000 7f4e61066700 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557fb6605d70 0x557fb6608190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.631+0000 7f4e61066700 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557fb6605d70 0x557fb6608190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45fc694964d8187c server_cookie=8123a2cc909e049f in_seq=0 out_seq=0 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.631+0000 7f4e457fa700 10 client.15639 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.631+0000 7f4e457fa700 1 -- 172.21.15.42:0/945549018 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fb65bbee0 con 0x557fb6605d70 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.631+0000 7f4e457fa700 10 client.15639 handle_command_reply: tid=0 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.631+0000 7f4e6fcc4b80 1 client.15639 shutdown 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.631+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/945549018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65ecac0 msgr2=0x557fb65f85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:05.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.631+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65ecac0 0x557fb65f85f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4e58005910 tx=0x7f4e580047c0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/945549018 shutdown_connections 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557fb6605d70 0x557fb6608190 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 0x557fb65f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e457fa700 0 client.15639 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65ecac0 0x557fb65f85f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/945549018 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f1d40 0x557fb65f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/945549018 >> 172.21.15.42:0/945549018 conn(0x557fb651b780 msgr2=0x557fb65eb570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/945549018 shutdown_connections 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/945549018 wait complete. 2023-12-12T09:59:05.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.632+0000 7f4e6fcc4b80 20 client.15639 trim_cache size 0 max 16384 2023-12-12T09:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.647+0000 7f4e6fcc4b80 1 Processor -- start 2023-12-12T09:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.647+0000 7f4e6fcc4b80 1 -- start start 2023-12-12T09:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.647+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 0x557fb65ebe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.647+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.647+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 0x557fb660cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.647+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb66033d0 con 0x557fb65f1d40 2023-12-12T09:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.647+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb65f6840 con 0x557fb65f2660 2023-12-12T09:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.647+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb65f0610 con 0x557fb65ecac0 2023-12-12T09:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.647+0000 7f4e61066700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61066700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65f82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35912/0 (socket says 172.21.15.42:35912) 2023-12-12T09:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61066700 1 -- 172.21.15.42:0/3953590527 learned_addr learned my addr 172.21.15.42:0/3953590527 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61867700 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 0x557fb65ebe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e62068700 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 0x557fb660cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61066700 1 -- 172.21.15.42:0/3953590527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 msgr2=0x557fb65ebe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61066700 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 0x557fb65ebe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61066700 1 -- 172.21.15.42:0/3953590527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 msgr2=0x557fb660cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61066700 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 0x557fb660cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61066700 1 -- 172.21.15.42:0/3953590527 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fb6352c00 con 0x557fb65f1d40 2023-12-12T09:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61867700 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 0x557fb65ebe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e61066700 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65f82b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c099860 tx=0x7f4e4c022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f1170dde804417 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e50d82700 1 -- 172.21.15.42:0/3953590527 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0cbae0 con 0x557fb65f1d40 2023-12-12T09:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e50d82700 1 -- 172.21.15.42:0/3953590527 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c09b510 con 0x557fb65f1d40 2023-12-12T09:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.648+0000 7f4e50d82700 1 -- 172.21.15.42:0/3953590527 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c066ee0 con 0x557fb65f1d40 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3953590527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 msgr2=0x557fb65f82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65f82b0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c099860 tx=0x7f4e4c022f90 comp rx=0 tx=0).stop 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3953590527 shutdown_connections 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65f2660 0x557fb660cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65ecac0 0x557fb65ebe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3953590527 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65f82b0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3953590527 >> 172.21.15.42:0/3953590527 conn(0x557fb651b780 msgr2=0x557fb65e86d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3953590527 shutdown_connections 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3953590527 wait complete. 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 -- start start 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.649+0000 7f4e6fcc4b80 1 Processor -- start 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65ecac0 0x557fb65fd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e6fcc4b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f2660 0x557fb65fb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb65fb920 con 0x557fb65f1d40 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb65fba90 con 0x557fb65ecac0 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e6fcc4b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fb665b250 con 0x557fb65f2660 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e6fcc4b80 10 client.0 _ll_register_callbacks cb 0x557fb6425a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61066700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61066700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65fad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35918/0 (socket says 172.21.15.42:35918) 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61066700 1 -- 172.21.15.42:0/3060192103 learned_addr learned my addr 172.21.15.42:0/3060192103 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e62068700 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f2660 0x557fb65fb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61867700 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65ecac0 0x557fb65fd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61066700 1 -- 172.21.15.42:0/3060192103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f2660 msgr2=0x557fb65fb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61066700 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f2660 0x557fb65fb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61066700 1 -- 172.21.15.42:0/3060192103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65ecac0 msgr2=0x557fb65fd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61066700 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65ecac0 0x557fb65fd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61066700 1 -- 172.21.15.42:0/3060192103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fb63bd5d0 con 0x557fb65f1d40 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61867700 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65ecac0 0x557fb65fd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e61066700 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65fad10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009ce0 tx=0x7f4e4c097980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02a9dad9b0514ba server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e457fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e457fa700 1 -- 172.21.15.42:0/3060192103 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0cbae0 con 0x557fb65f1d40 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e457fa700 1 -- 172.21.15.42:0/3060192103 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4e4c0228c0 con 0x557fb65f1d40 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.650+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3060192103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e4c09bc10 con 0x557fb65f1d40 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e457fa700 1 -- 172.21.15.42:0/3060192103 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e4c09aa00 con 0x557fb65f1d40 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e6fcc4b80 10 client.15645 fetch_fsmap learned FSMap version 27 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3060192103 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557fb63901f0 con 0x557fb65f1d40 2023-12-12T09:59:05.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e457fa700 1 -- 172.21.15.42:0/3060192103 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e4c09bc10 con 0x557fb65f1d40 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e457fa700 1 -- 172.21.15.42:0/3060192103 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4e4c0cbe80 con 0x557fb65f1d40 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e6fcc4b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e6fcc4b80 10 client.15645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e6fcc4b80 20 client.15645 populate_metadata read hostname 'smithi042' 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557fb665c4f0 0x557fb665e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e6fcc4b80 4 client.15645 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3060192103 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557fb6506f70 con 0x557fb665c4f0 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.651+0000 7f4e61867700 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557fb665c4f0 0x557fb665e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.652+0000 7f4e457fa700 10 client.15645 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:05.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.652+0000 7f4e61867700 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557fb665c4f0 0x557fb665e910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f57a40dc848b632c server_cookie=6cdccbbef4175a1f in_seq=0 out_seq=0 2023-12-12T09:59:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.659+0000 7f4e457fa700 1 -- 172.21.15.42:0/3060192103 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557fb6506f70 con 0x557fb665c4f0 2023-12-12T09:59:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.659+0000 7f4e457fa700 10 client.15645 handle_command_reply: tid=0 2023-12-12T09:59:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.659+0000 7f4e6fcc4b80 1 client.15645 shutdown 2023-12-12T09:59:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.659+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3060192103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 msgr2=0x557fb65fad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.659+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65fad10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009ce0 tx=0x7f4e4c097980 comp rx=0 tx=0).stop 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3060192103 shutdown_connections 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557fb665c4f0 0x557fb665e910 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557fb65ecac0 0x557fb65fd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557fb65f2660 0x557fb65fb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e6fcc4b80 1 --2- 172.21.15.42:0/3060192103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557fb65f1d40 0x557fb65fad10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e457fa700 0 client.15645 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3060192103 >> 172.21.15.42:0/3060192103 conn(0x557fb651b780 msgr2=0x557fb660cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3060192103 shutdown_connections 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e6fcc4b80 1 -- 172.21.15.42:0/3060192103 wait complete. 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e6fcc4b80 20 client.15645 trim_cache size 0 max 16384 2023-12-12T09:59:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T09:59:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T09:59:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T09:59:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T09:59:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 -- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e480a1710 msgr2=0x7f4e480a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e480a1710 0x7f4e480a3bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009c60 tx=0x7f4e58009260 comp rx=0 tx=0).stop 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 -- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c100d90 msgr2=0x7f4e5c19db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c100d90 0x7f4e5c19db10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005910 tx=0x7f4e4c004730 comp rx=0 tx=0).stop 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 -- 172.21.15.42:0/1402680803 shutdown_connections 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e480a1710 0x7f4e480a3bd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e5c106480 0x7f4e5c197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e5c1003d0 0x7f4e5c19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 --2- 172.21.15.42:0/1402680803 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e5c100d90 0x7f4e5c19db10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 -- 172.21.15.42:0/1402680803 >> 172.21.15.42:0/1402680803 conn(0x7f4e5c077570 msgr2=0x7f4e5c068c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 -- 172.21.15.42:0/1402680803 shutdown_connections 2023-12-12T09:59:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:05.660+0000 7f4e63acb700 1 -- 172.21.15.42:0/1402680803 wait complete. 2023-12-12T09:59:06.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:06 smithi111 ceph-mon[102565]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2023-12-12T09:59:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:06 smithi173 ceph-mon[103308]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2023-12-12T09:59:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:06 smithi042 ceph-mon[98894]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 29 op/s 2023-12-12T09:59:08.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:08 smithi111 ceph-mon[102565]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-12-12T09:59:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:08 smithi173 ceph-mon[103308]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-12-12T09:59:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:08 smithi042 ceph-mon[98894]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2023-12-12T09:59:10.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:10 smithi111 ceph-mon[102565]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2023-12-12T09:59:10.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:10 smithi173 ceph-mon[103308]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2023-12-12T09:59:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:10 smithi042 ceph-mon[98894]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 27 op/s 2023-12-12T09:59:12.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:12 smithi111 ceph-mon[102565]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-12-12T09:59:12.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:12 smithi173 ceph-mon[103308]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-12-12T09:59:12.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:12 smithi042 ceph-mon[98894]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 23 op/s 2023-12-12T09:59:14.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:14 smithi111 ceph-mon[102565]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 26 op/s 2023-12-12T09:59:14.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:14 smithi173 ceph-mon[103308]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 26 op/s 2023-12-12T09:59:14.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:14 smithi042 ceph-mon[98894]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 26 op/s 2023-12-12T09:59:16.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:16 smithi111 ceph-mon[102565]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 23 op/s 2023-12-12T09:59:16.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:16 smithi173 ceph-mon[103308]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 23 op/s 2023-12-12T09:59:16.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:16 smithi042 ceph-mon[98894]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 23 op/s 2023-12-12T09:59:18.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:18 smithi111 ceph-mon[102565]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-12-12T09:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.803+0000 7fea2cf96700 1 -- 172.21.15.42:0/2268661919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 msgr2=0x7fea2810d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.803+0000 7fea2cf96700 1 --2- 172.21.15.42:0/2268661919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 0x7fea2810d210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fea2000b0a0 tx=0x7fea2000b3b0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.804+0000 7fea2cf96700 1 -- 172.21.15.42:0/2268661919 shutdown_connections 2023-12-12T09:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.804+0000 7fea2cf96700 1 --2- 172.21.15.42:0/2268661919 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea28106570 0x7fea28106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.804+0000 7fea2cf96700 1 --2- 172.21.15.42:0/2268661919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 0x7fea2810d210 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.804+0000 7fea2cf96700 1 --2- 172.21.15.42:0/2268661919 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea281008d0 0x7fea28100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.804+0000 7fea2cf96700 1 -- 172.21.15.42:0/2268661919 >> 172.21.15.42:0/2268661919 conn(0x7fea28077240 msgr2=0x7fea28077650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.804+0000 7fea2cf96700 1 -- 172.21.15.42:0/2268661919 shutdown_connections 2023-12-12T09:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.804+0000 7fea2cf96700 1 -- 172.21.15.42:0/2268661919 wait complete. 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea2cf96700 1 Processor -- start 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea2cf96700 1 -- start start 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea2cf96700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea281008d0 0x7fea2819c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea2cf96700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 0x7fea2819cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea2cf96700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea28106570 0x7fea28196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea2cf96700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea28196e30 con 0x7fea28101290 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea2cf96700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea28196fa0 con 0x7fea28106570 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea2cf96700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea28197160 con 0x7fea281008d0 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 0x7fea2819cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 0x7fea2819cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35932/0 (socket says 172.21.15.42:35932) 2023-12-12T09:59:18.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea25d9b700 1 -- 172.21.15.42:0/464526185 learned_addr learned my addr 172.21.15.42:0/464526185 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea2659c700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea281008d0 0x7fea2819c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea26d9d700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea28106570 0x7fea28196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea25d9b700 1 -- 172.21.15.42:0/464526185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea281008d0 msgr2=0x7fea2819c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea25d9b700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea281008d0 0x7fea2819c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea25d9b700 1 -- 172.21.15.42:0/464526185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea28106570 msgr2=0x7fea28196780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea25d9b700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea28106570 0x7fea28196780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.805+0000 7fea25d9b700 1 -- 172.21.15.42:0/464526185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea20009d00 con 0x7fea28101290 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.806+0000 7fea25d9b700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 0x7fea2819cb70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00eab0 tx=0x7fea1c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.806+0000 7fea177fe700 1 -- 172.21.15.42:0/464526185 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c013070 con 0x7fea28101290 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.806+0000 7fea2cf96700 1 -- 172.21.15.42:0/464526185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea28197440 con 0x7fea28101290 2023-12-12T09:59:18.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.806+0000 7fea177fe700 1 -- 172.21.15.42:0/464526185 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea1c00be80 con 0x7fea28101290 2023-12-12T09:59:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.806+0000 7fea177fe700 1 -- 172.21.15.42:0/464526185 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0163b0 con 0x7fea28101290 2023-12-12T09:59:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.806+0000 7fea2cf96700 1 -- 172.21.15.42:0/464526185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea28069970 con 0x7fea28101290 2023-12-12T09:59:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.809+0000 7fea177fe700 1 -- 172.21.15.42:0/464526185 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136239+0+0 (secure 0 0 0) 0x7fea1c0175a0 con 0x7fea28101290 2023-12-12T09:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.810+0000 7fea177fe700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea100a1710 0x7fea100a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.810+0000 7fea177fe700 1 -- 172.21.15.42:0/464526185 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fea1c0cecb0 con 0x7fea28101290 2023-12-12T09:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.810+0000 7fea2659c700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea100a1710 0x7fea100a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.811+0000 7fea2659c700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea100a1710 0x7fea100a3bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea18005950 tx=0x7fea1800b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.811+0000 7fea3918fb80 1 Processor -- start 2023-12-12T09:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.811+0000 7fea3918fb80 1 -- start start 2023-12-12T09:59:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.811+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 0x56417811b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.811+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811b660 0x564178115580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.811+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x564178115f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea3918fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564178070af0 con 0x56417811b660 2023-12-12T09:59:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea3918fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417806e410 con 0x564178115ac0 2023-12-12T09:59:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea3918fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x564178069d00 con 0x56417811ad40 2023-12-12T09:59:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811b660 0x564178115580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811b660 0x564178115580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35942/0 (socket says 172.21.15.42:35942) 2023-12-12T09:59:18.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea25d9b700 1 -- 172.21.15.42:0/1495797715 learned_addr learned my addr 172.21.15.42:0/1495797715 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea26d9d700 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x564178115f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea2659c700 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 0x56417811b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea25d9b700 1 -- 172.21.15.42:0/1495797715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 msgr2=0x56417811b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea25d9b700 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 0x56417811b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea25d9b700 1 -- 172.21.15.42:0/1495797715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 msgr2=0x564178115f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea25d9b700 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x564178115f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea25d9b700 1 -- 172.21.15.42:0/1495797715 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564177ee65d0 con 0x56417811b660 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea26d9d700 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x564178115f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.812+0000 7fea25d9b700 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811b660 0x564178115580 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b5a0 tx=0x7fea1c09b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a236657639a769b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea14ff9700 1 -- 172.21.15.42:0/1495797715 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0bb0c0 con 0x56417811b660 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea14ff9700 1 -- 172.21.15.42:0/1495797715 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea1c09b050 con 0x56417811b660 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea14ff9700 1 -- 172.21.15.42:0/1495797715 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c09ae40 con 0x56417811b660 2023-12-12T09:59:18.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea3918fb80 1 -- 172.21.15.42:0/1495797715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811b660 msgr2=0x564178115580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea3918fb80 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811b660 0x564178115580 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00b5a0 tx=0x7fea1c09b540 comp rx=0 tx=0).stop 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea3918fb80 1 -- 172.21.15.42:0/1495797715 shutdown_connections 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea3918fb80 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x564178115f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea3918fb80 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 0x56417811b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea3918fb80 1 --2- 172.21.15.42:0/1495797715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811b660 0x564178115580 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.813+0000 7fea3918fb80 1 -- 172.21.15.42:0/1495797715 >> 172.21.15.42:0/1495797715 conn(0x564178044780 msgr2=0x56417810fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 -- 172.21.15.42:0/1495797715 shutdown_connections 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 -- 172.21.15.42:0/1495797715 wait complete. 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 -- start start 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 Processor -- start 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564178115ac0 0x564178121650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 0x56417811e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 0x56417811f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564178023500 con 0x564178115ac0 2023-12-12T09:59:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641780231a0 con 0x56417811b660 2023-12-12T09:59:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.814+0000 7fea3918fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x564177ee88c0 con 0x56417811ad40 2023-12-12T09:59:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea3918fb80 10 client.0 _ll_register_callbacks cb 0x564177f4ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:59:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea2659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564178115ac0 0x564178121650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea2659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564178115ac0 0x564178121650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35944/0 (socket says 172.21.15.42:35944) 2023-12-12T09:59:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea2659c700 1 -- 172.21.15.42:0/3475948335 learned_addr learned my addr 172.21.15.42:0/3475948335 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea25d9b700 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 0x56417811e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea26d9d700 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 0x56417811f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea2659c700 1 -- 172.21.15.42:0/3475948335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 msgr2=0x56417811e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea2659c700 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 0x56417811e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea2659c700 1 -- 172.21.15.42:0/3475948335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 msgr2=0x56417811f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea2659c700 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 0x56417811f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea2659c700 1 -- 172.21.15.42:0/3475948335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2810db40 con 0x564178115ac0 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea25d9b700 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 0x56417811e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea2659c700 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564178115ac0 0x564178121650 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fea180116c0 tx=0x7fea180047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ad29bede6d886c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea0dffb700 1 -- 172.21.15.42:0/3475948335 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18004a30 con 0x564178115ac0 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea0dffb700 1 -- 172.21.15.42:0/3475948335 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fea18017070 con 0x564178115ac0 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.815+0000 7fea3918fb80 1 -- 172.21.15.42:0/3475948335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564178108730 con 0x564178115ac0 2023-12-12T09:59:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea0dffb700 1 -- 172.21.15.42:0/3475948335 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea18004b90 con 0x564178115ac0 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea3918fb80 10 client.15657 fetch_fsmap learned FSMap version 27 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea3918fb80 1 -- 172.21.15.42:0/3475948335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fea28069970 con 0x564178115ac0 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea0dffb700 1 -- 172.21.15.42:0/3475948335 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564178108730 con 0x564178115ac0 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea0dffb700 1 -- 172.21.15.42:0/3475948335 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea1801f530 con 0x564178115ac0 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea3918fb80 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea3918fb80 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea3918fb80 20 client.15657 populate_metadata read hostname 'smithi042' 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea3918fb80 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56417812f0d0 0x5641781314f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea3918fb80 4 client.15657 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea3918fb80 1 -- 172.21.15.42:0/3475948335 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641780e4ee0 con 0x56417812f0d0 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.816+0000 7fea25d9b700 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56417812f0d0 0x5641781314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.817+0000 7fea0dffb700 10 client.15657 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.817+0000 7fea25d9b700 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56417812f0d0 0x5641781314f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51118d54ea66b353 server_cookie=7dcfdf96bd82aff2 in_seq=0 out_seq=0 2023-12-12T09:59:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea0dffb700 1 -- 172.21.15.42:0/3475948335 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5641780e4ee0 con 0x56417812f0d0 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea0dffb700 10 client.15657 handle_command_reply: tid=0 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 client.15657 shutdown 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 -- 172.21.15.42:0/3475948335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564178115ac0 msgr2=0x564178121650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564178115ac0 0x564178121650 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fea180116c0 tx=0x7fea180047c0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 -- 172.21.15.42:0/3475948335 shutdown_connections 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56417812f0d0 0x5641781314f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 0x56417811f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564178115ac0 0x564178121650 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 --2- 172.21.15.42:0/3475948335 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811ad40 0x56417811e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 -- 172.21.15.42:0/3475948335 >> 172.21.15.42:0/3475948335 conn(0x564178044780 msgr2=0x564178114570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea0dffb700 0 client.15657 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 -- 172.21.15.42:0/3475948335 shutdown_connections 2023-12-12T09:59:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 1 -- 172.21.15.42:0/3475948335 wait complete. 2023-12-12T09:59:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.818+0000 7fea3918fb80 20 client.15657 trim_cache size 0 max 16384 2023-12-12T09:59:18.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:18 smithi173 ceph-mon[103308]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-12-12T09:59:18.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.847+0000 7fea3918fb80 1 Processor -- start 2023-12-12T09:59:18.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.847+0000 7fea3918fb80 1 -- start start 2023-12-12T09:59:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.847+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564178115ac0 0x564178126040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 0x564178126580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 0x56417811d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea3918fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56417812bba0 con 0x56417811ad40 2023-12-12T09:59:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea3918fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x564178108700 con 0x56417811b660 2023-12-12T09:59:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea3918fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x564178110d70 con 0x564178115ac0 2023-12-12T09:59:18.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 0x564178126580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea25d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 0x564178126580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35954/0 (socket says 172.21.15.42:35954) 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea25d9b700 1 -- 172.21.15.42:0/928083309 learned_addr learned my addr 172.21.15.42:0/928083309 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea2659c700 1 --2- 172.21.15.42:0/928083309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564178115ac0 0x564178126040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea26d9d700 1 --2- 172.21.15.42:0/928083309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 0x56417811d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea25d9b700 1 -- 172.21.15.42:0/928083309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564178115ac0 msgr2=0x564178126040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea25d9b700 1 --2- 172.21.15.42:0/928083309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564178115ac0 0x564178126040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea25d9b700 1 -- 172.21.15.42:0/928083309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 msgr2=0x56417811d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea25d9b700 1 --2- 172.21.15.42:0/928083309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 0x56417811d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea25d9b700 1 -- 172.21.15.42:0/928083309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564177e92d80 con 0x56417811ad40 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea25d9b700 1 --2- 172.21.15.42:0/928083309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 0x564178126580 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fea1c09b3f0 tx=0x7fea1c017410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ddcb0452ffef549 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:18.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.848+0000 7fea157fa700 1 -- 172.21.15.42:0/928083309 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c09bdf0 con 0x56417811ad40 2023-12-12T09:59:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.849+0000 7fea157fa700 1 -- 172.21.15.42:0/928083309 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea1c09f840 con 0x56417811ad40 2023-12-12T09:59:18.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.849+0000 7fea157fa700 1 -- 172.21.15.42:0/928083309 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c09ae40 con 0x56417811ad40 2023-12-12T09:59:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.849+0000 7fea3918fb80 1 -- 172.21.15.42:0/928083309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 msgr2=0x564178126580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.849+0000 7fea3918fb80 1 --2- 172.21.15.42:0/928083309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 0x564178126580 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fea1c09b3f0 tx=0x7fea1c017410 comp rx=0 tx=0).stop 2023-12-12T09:59:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.849+0000 7fea3918fb80 1 -- 172.21.15.42:0/928083309 shutdown_connections 2023-12-12T09:59:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.849+0000 7fea3918fb80 1 --2- 172.21.15.42:0/928083309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56417811b660 0x56417811d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.849+0000 7fea3918fb80 1 --2- 172.21.15.42:0/928083309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x564178115ac0 0x564178126040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.849+0000 7fea3918fb80 1 --2- 172.21.15.42:0/928083309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 0x564178126580 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.849+0000 7fea3918fb80 1 -- 172.21.15.42:0/928083309 >> 172.21.15.42:0/928083309 conn(0x564178044780 msgr2=0x56417812aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 -- 172.21.15.42:0/928083309 shutdown_connections 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 -- 172.21.15.42:0/928083309 wait complete. 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 -- start start 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 Processor -- start 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x5641781201e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 0x564178120720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811b660 0x5641781845d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x564178121030 con 0x56417811ad40 2023-12-12T09:59:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x564178184b10 con 0x564178115ac0 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x564178184c80 con 0x56417811b660 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.850+0000 7fea3918fb80 10 client.0 _ll_register_callbacks cb 0x7fea18010d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea2659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x5641781201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea2659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x5641781201e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39018/0 (socket says 172.21.15.42:39018) 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea2659c700 1 -- 172.21.15.42:0/2502963682 learned_addr learned my addr 172.21.15.42:0/2502963682 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea26d9d700 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811b660 0x5641781845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea2659c700 1 -- 172.21.15.42:0/2502963682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811b660 msgr2=0x5641781845d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea2659c700 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811b660 0x5641781845d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea2659c700 1 -- 172.21.15.42:0/2502963682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 msgr2=0x564178120720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea2659c700 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 0x564178120720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea2659c700 1 -- 172.21.15.42:0/2502963682 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564177ee65d0 con 0x564178115ac0 2023-12-12T09:59:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea2659c700 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x5641781201e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fea180116c0 tx=0x7fea1801f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=238758108733e38 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea0dffb700 1 -- 172.21.15.42:0/2502963682 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1800b980 con 0x564178115ac0 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea3918fb80 1 -- 172.21.15.42:0/2502963682 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea1c09b050 con 0x564178115ac0 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.851+0000 7fea0dffb700 1 -- 172.21.15.42:0/2502963682 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fea18017070 con 0x564178115ac0 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea0dffb700 1 -- 172.21.15.42:0/2502963682 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea18011b30 con 0x564178115ac0 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea3918fb80 10 client.25355 fetch_fsmap learned FSMap version 27 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea3918fb80 1 -- 172.21.15.42:0/2502963682 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564177e7bc00 con 0x564178115ac0 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea0dffb700 1 -- 172.21.15.42:0/2502963682 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea1c09b050 con 0x564178115ac0 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea0dffb700 1 -- 172.21.15.42:0/2502963682 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fea180054b0 con 0x564178115ac0 2023-12-12T09:59:18.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea3918fb80 10 client.25355 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:59:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea3918fb80 10 client.25355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:59:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea3918fb80 20 client.25355 populate_metadata read hostname 'smithi042' 2023-12-12T09:59:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea3918fb80 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5641781865c0 0x5641781889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea3918fb80 4 client.25355 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T09:59:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea3918fb80 1 -- 172.21.15.42:0/2502963682 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56417802ff70 con 0x5641781865c0 2023-12-12T09:59:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.852+0000 7fea25d9b700 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5641781865c0 0x5641781889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.853+0000 7fea25d9b700 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5641781865c0 0x5641781889e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb31c6f6b53707ab server_cookie=953a9be320b89d3b in_seq=0 out_seq=0 2023-12-12T09:59:18.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.853+0000 7fea0dffb700 10 client.25355 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea0dffb700 1 -- 172.21.15.42:0/2502963682 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56417802ff70 con 0x5641781865c0 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea0dffb700 10 client.25355 handle_command_reply: tid=0 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea3918fb80 1 client.25355 shutdown 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea3918fb80 1 -- 172.21.15.42:0/2502963682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 msgr2=0x5641781201e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea3918fb80 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x5641781201e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fea180116c0 tx=0x7fea1801f710 comp rx=0 tx=0).stop 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea3918fb80 1 -- 172.21.15.42:0/2502963682 shutdown_connections 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea3918fb80 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5641781865c0 0x5641781889e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea3918fb80 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x564178115ac0 0x5641781201e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea3918fb80 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56417811b660 0x5641781845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea3918fb80 1 --2- 172.21.15.42:0/2502963682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56417811ad40 0x564178120720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea3918fb80 1 -- 172.21.15.42:0/2502963682 >> 172.21.15.42:0/2502963682 conn(0x564178044780 msgr2=0x56417812aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:18.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.860+0000 7fea0dffb700 0 client.25355 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea3918fb80 1 -- 172.21.15.42:0/2502963682 shutdown_connections 2023-12-12T09:59:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea3918fb80 1 -- 172.21.15.42:0/2502963682 wait complete. 2023-12-12T09:59:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea3918fb80 20 client.25355 trim_cache size 0 max 16384 2023-12-12T09:59:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T09:59:18.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T09:59:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T09:59:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T09:59:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 -- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea100a1710 msgr2=0x7fea100a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea100a1710 0x7fea100a3bd0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fea18005950 tx=0x7fea1800b410 comp rx=0 tx=0).stop 2023-12-12T09:59:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 -- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 msgr2=0x7fea2819cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 0x7fea2819cb70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00eab0 tx=0x7fea1c00edc0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 -- 172.21.15.42:0/464526185 shutdown_connections 2023-12-12T09:59:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea100a1710 0x7fea100a3bd0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea28106570 0x7fea28196780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea281008d0 0x7fea2819c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 --2- 172.21.15.42:0/464526185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea28101290 0x7fea2819cb70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.861+0000 7fea2cf96700 1 -- 172.21.15.42:0/464526185 >> 172.21.15.42:0/464526185 conn(0x7fea28077240 msgr2=0x7fea280ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.862+0000 7fea2cf96700 1 -- 172.21.15.42:0/464526185 shutdown_connections 2023-12-12T09:59:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:18.862+0000 7fea2cf96700 1 -- 172.21.15.42:0/464526185 wait complete. 2023-12-12T09:59:18.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:18 smithi042 ceph-mon[98894]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2023-12-12T09:59:20.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:20 smithi111 ceph-mon[102565]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-12-12T09:59:20.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:20 smithi173 ceph-mon[103308]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-12-12T09:59:20.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:20 smithi042 ceph-mon[98894]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 2.5 KiB/s wr, 14 op/s 2023-12-12T09:59:22.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:22 smithi111 ceph-mon[102565]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-12-12T09:59:22.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:22 smithi173 ceph-mon[103308]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-12-12T09:59:22.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:22 smithi042 ceph-mon[98894]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2023-12-12T09:59:24.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:24 smithi111 ceph-mon[102565]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-12-12T09:59:24.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:24 smithi173 ceph-mon[103308]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-12-12T09:59:24.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:24 smithi042 ceph-mon[98894]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2023-12-12T09:59:26.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:26 smithi173 ceph-mon[103308]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-12-12T09:59:26.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:26 smithi042 ceph-mon[98894]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-12-12T09:59:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:26 smithi111 ceph-mon[102565]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2023-12-12T09:59:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:28 smithi173 ceph-mon[103308]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-12T09:59:28.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:28 smithi042 ceph-mon[98894]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-12T09:59:28.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:28 smithi111 ceph-mon[102565]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-12T09:59:30.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:30 smithi173 ceph-mon[103308]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-12-12T09:59:30.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:30 smithi042 ceph-mon[98894]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-12-12T09:59:30.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:30 smithi111 ceph-mon[102565]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.030+0000 7fce99b05700 1 -- 172.21.15.42:0/4157903552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 msgr2=0x7fce94100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.030+0000 7fce99b05700 1 --2- 172.21.15.42:0/4157903552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 0x7fce94100fd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fce88009990 tx=0x7fce88009ca0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.031+0000 7fce99b05700 1 -- 172.21.15.42:0/4157903552 shutdown_connections 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.031+0000 7fce99b05700 1 --2- 172.21.15.42:0/4157903552 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce94101510 0x7fce9410d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.031+0000 7fce99b05700 1 --2- 172.21.15.42:0/4157903552 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce941053b0 0x7fce94105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.031+0000 7fce99b05700 1 --2- 172.21.15.42:0/4157903552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 0x7fce94100fd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.031+0000 7fce99b05700 1 -- 172.21.15.42:0/4157903552 >> 172.21.15.42:0/4157903552 conn(0x7fce940769d0 msgr2=0x7fce940fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.031+0000 7fce99b05700 1 -- 172.21.15.42:0/4157903552 shutdown_connections 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.031+0000 7fce99b05700 1 -- 172.21.15.42:0/4157903552 wait complete. 2023-12-12T09:59:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.031+0000 7fce99b05700 1 Processor -- start 2023-12-12T09:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce99b05700 1 -- start start 2023-12-12T09:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce99b05700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 0x7fce9410db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce99b05700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce94101510 0x7fce9410e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce99b05700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce941053b0 0x7fce9410e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce99b05700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce941a5d80 con 0x7fce94100b50 2023-12-12T09:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce99b05700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce941a5ef0 con 0x7fce94101510 2023-12-12T09:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce99b05700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce941a60b0 con 0x7fce941053b0 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 0x7fce9410db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 0x7fce9410db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35972/0 (socket says 172.21.15.42:35972) 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce937fe700 1 -- 172.21.15.42:0/3861212381 learned_addr learned my addr 172.21.15.42:0/3861212381 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce93fff700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce941053b0 0x7fce9410e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.032+0000 7fce92ffd700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce94101510 0x7fce9410e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce937fe700 1 -- 172.21.15.42:0/3861212381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce941053b0 msgr2=0x7fce9410e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce937fe700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce941053b0 0x7fce9410e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce937fe700 1 -- 172.21.15.42:0/3861212381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce94101510 msgr2=0x7fce9410e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce937fe700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce94101510 0x7fce9410e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce937fe700 1 -- 172.21.15.42:0/3861212381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce880095f0 con 0x7fce94100b50 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce92ffd700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce94101510 0x7fce9410e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce937fe700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 0x7fce9410db30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fce8400d980 tx=0x7fce8400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce90ff9700 1 -- 172.21.15.42:0/3861212381 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8400bb60 con 0x7fce94100b50 2023-12-12T09:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce99b05700 1 -- 172.21.15.42:0/3861212381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce941a6390 con 0x7fce94100b50 2023-12-12T09:59:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce90ff9700 1 -- 172.21.15.42:0/3861212381 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce84007640 con 0x7fce94100b50 2023-12-12T09:59:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce90ff9700 1 -- 172.21.15.42:0/3861212381 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84004e90 con 0x7fce94100b50 2023-12-12T09:59:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.033+0000 7fce99b05700 1 -- 172.21.15.42:0/3861212381 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce941a68f0 con 0x7fce94100b50 2023-12-12T09:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.037+0000 7fce90ff9700 1 -- 172.21.15.42:0/3861212381 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136239+0+0 (secure 0 0 0) 0x7fce8400bcc0 con 0x7fce94100b50 2023-12-12T09:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.037+0000 7fce90ff9700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce7c0a1710 0x7fce7c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.038+0000 7fce90ff9700 1 -- 172.21.15.42:0/3861212381 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fce840cd5c0 con 0x7fce94100b50 2023-12-12T09:59:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.038+0000 7fcea5cfeb80 1 Processor -- start 2023-12-12T09:59:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.038+0000 7fce92ffd700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce7c0a1710 0x7fce7c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.038+0000 7fcea5cfeb80 1 -- start start 2023-12-12T09:59:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.038+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4550 0x5615e37c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.038+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37bed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37bf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e3711af0 con 0x5615e37c4e70 2023-12-12T09:59:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e370f410 con 0x5615e37c4550 2023-12-12T09:59:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e370ad00 con 0x5615e37bf320 2023-12-12T09:59:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37bed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37bed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35982/0 (socket says 172.21.15.42:35982) 2023-12-12T09:59:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 -- 172.21.15.42:0/774244239 learned_addr learned my addr 172.21.15.42:0/774244239 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:32.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce7c0a1710 0x7fce7c0a3bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fce8800b5c0 tx=0x7fce88005fb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 -- 172.21.15.42:0/774244239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 msgr2=0x5615e37bf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 --2- 172.21.15.42:0/774244239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37bf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 -- 172.21.15.42:0/774244239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4550 msgr2=0x5615e37c4930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-12T09:59:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 --2- 172.21.15.42:0/774244239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4550 0x5615e37c4930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 -- 172.21.15.42:0/774244239 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615e35875d0 con 0x5615e37c4e70 2023-12-12T09:59:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.039+0000 7fce92ffd700 1 --2- 172.21.15.42:0/774244239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37bed90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fce8800bf80 tx=0x7fce88020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb415f087490bd0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.040+0000 7fce79ffb700 1 -- 172.21.15.42:0/774244239 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88003b20 con 0x5615e37c4e70 2023-12-12T09:59:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.040+0000 7fce79ffb700 1 -- 172.21.15.42:0/774244239 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce8801d040 con 0x5615e37c4e70 2023-12-12T09:59:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.040+0000 7fce79ffb700 1 -- 172.21.15.42:0/774244239 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8800f880 con 0x5615e37c4e70 2023-12-12T09:59:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/774244239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 msgr2=0x5615e37bed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/774244239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37bed90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fce8800bf80 tx=0x7fce88020000 comp rx=0 tx=0).stop 2023-12-12T09:59:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/774244239 shutdown_connections 2023-12-12T09:59:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/774244239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4550 0x5615e37c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/774244239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37bf7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/774244239 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37bed90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/774244239 >> 172.21.15.42:0/774244239 conn(0x5615e36e5780 msgr2=0x5615e37b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/774244239 shutdown_connections 2023-12-12T09:59:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/774244239 wait complete. 2023-12-12T09:59:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.041+0000 7fcea5cfeb80 1 -- start start 2023-12-12T09:59:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fcea5cfeb80 1 Processor -- start 2023-12-12T09:59:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37c8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4550 0x5615e37c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37bda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e36c4500 con 0x5615e37c4550 2023-12-12T09:59:32.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e36c41a0 con 0x5615e37c4e70 2023-12-12T09:59:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e35898c0 con 0x5615e37bf320 2023-12-12T09:59:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fcea5cfeb80 10 client.0 _ll_register_callbacks cb 0x5615e35efa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:59:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fce93fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37bda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fce93fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37bda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39050/0 (socket says 172.21.15.42:39050) 2023-12-12T09:59:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fce93fff700 1 -- 172.21.15.42:0/715463122 learned_addr learned my addr 172.21.15.42:0/715463122 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:32.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.042+0000 7fce937fe700 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce92ffd700 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4550 0x5615e37c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce93fff700 1 -- 172.21.15.42:0/715463122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 msgr2=0x5615e37c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce93fff700 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce93fff700 1 -- 172.21.15.42:0/715463122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4550 msgr2=0x5615e37c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce93fff700 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4550 0x5615e37c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce93fff700 1 -- 172.21.15.42:0/715463122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce940ffc00 con 0x5615e37c4e70 2023-12-12T09:59:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce92ffd700 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4550 0x5615e37c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:59:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce937fe700 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:59:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce93fff700 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37bda30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0077a0 tx=0x7fce8c007ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bdb36e51a76619a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T09:59:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce72ffd700 1 -- 172.21.15.42:0/715463122 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c0147e0 con 0x5615e37c4e70 2023-12-12T09:59:32.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/715463122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5615e37a9730 con 0x5615e37c4e70 2023-12-12T09:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.043+0000 7fce72ffd700 1 -- 172.21.15.42:0/715463122 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fce8c014940 con 0x5615e37c4e70 2023-12-12T09:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fce72ffd700 1 -- 172.21.15.42:0/715463122 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce8c0093f0 con 0x5615e37c4e70 2023-12-12T09:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fcea5cfeb80 10 client.25361 fetch_fsmap learned FSMap version 27 2023-12-12T09:59:32.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/715463122 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5615e3533d80 con 0x5615e37c4e70 2023-12-12T09:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fce72ffd700 1 -- 172.21.15.42:0/715463122 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5615e37a9730 con 0x5615e37c4e70 2023-12-12T09:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fce72ffd700 1 -- 172.21.15.42:0/715463122 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fce8c01f3b0 con 0x5615e37c4e70 2023-12-12T09:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fcea5cfeb80 10 client.25361 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:59:32.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fcea5cfeb80 10 client.25361 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fcea5cfeb80 20 client.25361 populate_metadata read hostname 'smithi042' 2023-12-12T09:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5615e37e1280 0x5615e37e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fcea5cfeb80 4 client.25361 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/715463122 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5615e3785ee0 con 0x5615e37e1280 2023-12-12T09:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.044+0000 7fce937fe700 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5615e37e1280 0x5615e37e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.045+0000 7fce72ffd700 10 client.25361 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.045+0000 7fce937fe700 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5615e37e1280 0x5615e37e36a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90c771b8e8daea87 server_cookie=71f49c9b9cda9495 in_seq=0 out_seq=0 2023-12-12T09:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.045+0000 7fce72ffd700 1 -- 172.21.15.42:0/715463122 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5615e3785ee0 con 0x5615e37e1280 2023-12-12T09:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.045+0000 7fce72ffd700 10 client.25361 handle_command_reply: tid=0 2023-12-12T09:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 client.25361 shutdown 2023-12-12T09:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/715463122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 msgr2=0x5615e37bda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37bda30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fce8c0077a0 tx=0x7fce8c007ab0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/715463122 shutdown_connections 2023-12-12T09:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5615e37e1280 0x5615e37e36a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37bda30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/715463122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4550 0x5615e37c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/715463122 >> 172.21.15.42:0/715463122 conn(0x5615e36e5780 msgr2=0x5615e37b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fce72ffd700 0 client.25361 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/715463122 shutdown_connections 2023-12-12T09:59:32.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/715463122 wait complete. 2023-12-12T09:59:32.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.046+0000 7fcea5cfeb80 20 client.25361 trim_cache size 0 max 16384 2023-12-12T09:59:32.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.075+0000 7fcea5cfeb80 1 Processor -- start 2023-12-12T09:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.075+0000 7fcea5cfeb80 1 -- start start 2023-12-12T09:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.075+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37bf320 0x5615e37e3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.075+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37c4550 0x5615e37b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.075+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37e7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e37beae0 con 0x5615e37bf320 2023-12-12T09:59:32.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e37c9d60 con 0x5615e37c4e70 2023-12-12T09:59:32.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e37a9700 con 0x5615e37c4550 2023-12-12T09:59:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce92ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37c4550 0x5615e37b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37bf320 0x5615e37e3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce937fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37bf320 0x5615e37e3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35994/0 (socket says 172.21.15.42:35994) 2023-12-12T09:59:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce937fe700 1 -- 172.21.15.42:0/2805179988 learned_addr learned my addr 172.21.15.42:0/2805179988 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce92ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37c4550 0x5615e37b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:50980/0 (socket says 172.21.15.42:50980) 2023-12-12T09:59:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce93fff700 1 --2- 172.21.15.42:0/2805179988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37e7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce937fe700 1 -- 172.21.15.42:0/2805179988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37c4550 msgr2=0x5615e37b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce937fe700 1 --2- 172.21.15.42:0/2805179988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37c4550 0x5615e37b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce937fe700 1 -- 172.21.15.42:0/2805179988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 msgr2=0x5615e37e7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce937fe700 1 --2- 172.21.15.42:0/2805179988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37e7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce937fe700 1 -- 172.21.15.42:0/2805179988 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5615e351cc00 con 0x5615e37bf320 2023-12-12T09:59:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.076+0000 7fce93fff700 1 --2- 172.21.15.42:0/2805179988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:59:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.077+0000 7fce937fe700 1 --2- 172.21.15.42:0/2805179988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37bf320 0x5615e37e3290 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fce840ca950 tx=0x7fce84007530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5917d0b2f4595d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.077+0000 7fce7a7fc700 1 -- 172.21.15.42:0/2805179988 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8409e150 con 0x5615e37bf320 2023-12-12T09:59:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.077+0000 7fce7a7fc700 1 -- 172.21.15.42:0/2805179988 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce840a2390 con 0x5615e37bf320 2023-12-12T09:59:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.077+0000 7fce7a7fc700 1 -- 172.21.15.42:0/2805179988 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce840994f0 con 0x5615e37bf320 2023-12-12T09:59:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2805179988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37bf320 msgr2=0x5615e37e3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2805179988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37bf320 0x5615e37e3290 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fce840ca950 tx=0x7fce84007530 comp rx=0 tx=0).stop 2023-12-12T09:59:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2805179988 shutdown_connections 2023-12-12T09:59:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2805179988 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4e70 0x5615e37e7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2805179988 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37bf320 0x5615e37e3290 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2805179988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37c4550 0x5615e37b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2805179988 >> 172.21.15.42:0/2805179988 conn(0x5615e36e5780 msgr2=0x5615e37c7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2805179988 shutdown_connections 2023-12-12T09:59:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2805179988 wait complete. 2023-12-12T09:59:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.078+0000 7fcea5cfeb80 1 -- start start 2023-12-12T09:59:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fcea5cfeb80 1 Processor -- start 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4550 0x5615e37b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fcea5cfeb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e3836740 con 0x5615e37c4e70 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e3836880 con 0x5615e37c4550 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fcea5cfeb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5615e38369c0 con 0x5615e37bf320 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fcea5cfeb80 10 client.0 _ll_register_callbacks cb 0x5615e35efa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fce93fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fce93fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37b8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36004/0 (socket says 172.21.15.42:36004) 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fce93fff700 1 -- 172.21.15.42:0/2981832826 learned_addr learned my addr 172.21.15.42:0/2981832826 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.079+0000 7fce937fe700 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce93fff700 1 -- 172.21.15.42:0/2981832826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 msgr2=0x5615e37b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce93fff700 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce93fff700 1 -- 172.21.15.42:0/2981832826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4550 msgr2=0x5615e37b8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce93fff700 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4550 0x5615e37b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce93fff700 1 -- 172.21.15.42:0/2981832826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5615e3533d80 con 0x5615e37c4e70 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce93fff700 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37b8e50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009710 tx=0x7fce8c01f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84402e80925b85d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce72ffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce72ffd700 1 -- 172.21.15.42:0/2981832826 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c01f980 con 0x5615e37c4e70 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce72ffd700 1 -- 172.21.15.42:0/2981832826 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fce8c01fae0 con 0x5615e37c4e70 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2981832826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce8801d040 con 0x5615e37c4e70 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce72ffd700 1 -- 172.21.15.42:0/2981832826 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce8c01fca0 con 0x5615e37c4e70 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fcea5cfeb80 10 client.15678 fetch_fsmap learned FSMap version 27 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2981832826 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fce941a68f0 con 0x5615e37c4e70 2023-12-12T09:59:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.080+0000 7fce72ffd700 1 -- 172.21.15.42:0/2981832826 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce8801d040 con 0x5615e37c4e70 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fce72ffd700 1 -- 172.21.15.42:0/2981832826 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fce8c005040 con 0x5615e37c4e70 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fcea5cfeb80 10 client.15678 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fcea5cfeb80 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fcea5cfeb80 20 client.15678 populate_metadata read hostname 'smithi042' 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5615e3838260 0x5615e383a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fcea5cfeb80 4 client.15678 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2981832826 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5615e36d0f70 con 0x5615e3838260 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fce937fe700 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5615e3838260 0x5615e383a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fce72ffd700 10 client.15678 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.081+0000 7fce937fe700 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5615e3838260 0x5615e383a680 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6e30e3d8dee5f10 server_cookie=7c9fe4fb5866ab47 in_seq=0 out_seq=0 2023-12-12T09:59:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.085+0000 7fce72ffd700 1 -- 172.21.15.42:0/2981832826 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5615e36d0f70 con 0x5615e3838260 2023-12-12T09:59:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.085+0000 7fce72ffd700 10 client.15678 handle_command_reply: tid=0 2023-12-12T09:59:32.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.085+0000 7fcea5cfeb80 1 client.15678 shutdown 2023-12-12T09:59:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2981832826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 msgr2=0x5615e37b8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37b8e50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009710 tx=0x7fce8c01f5e0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2981832826 shutdown_connections 2023-12-12T09:59:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5615e37c4e70 0x5615e37b8e50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5615e37bf320 0x5615e37b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5615e3838260 0x5615e383a680 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 --2- 172.21.15.42:0/2981832826 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5615e37c4550 0x5615e37b8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2981832826 >> 172.21.15.42:0/2981832826 conn(0x5615e36e5780 msgr2=0x5615e37b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:32.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fce72ffd700 0 client.15678 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2981832826 shutdown_connections 2023-12-12T09:59:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 1 -- 172.21.15.42:0/2981832826 wait complete. 2023-12-12T09:59:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.086+0000 7fcea5cfeb80 20 client.15678 trim_cache size 0 max 16384 2023-12-12T09:59:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T09:59:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T09:59:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T09:59:32.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 -- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce7c0a1710 msgr2=0x7fce7c0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce7c0a1710 0x7fce7c0a3bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fce8800b5c0 tx=0x7fce88005fb0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 -- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 msgr2=0x7fce9410db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 0x7fce9410db30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fce8400d980 tx=0x7fce8400dd40 comp rx=0 tx=0).stop 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 -- 172.21.15.42:0/3861212381 shutdown_connections 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce941053b0 0x7fce9410e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce94100b50 0x7fce9410db30 secure :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fce8400d980 tx=0x7fce8400dd40 comp rx=0 tx=0).stop 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce7c0a1710 0x7fce7c0a3bd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 --2- 172.21.15.42:0/3861212381 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce94101510 0x7fce9410e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:32.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 -- 172.21.15.42:0/3861212381 >> 172.21.15.42:0/3861212381 conn(0x7fce940769d0 msgr2=0x7fce940fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 -- 172.21.15.42:0/3861212381 shutdown_connections 2023-12-12T09:59:32.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:32.087+0000 7fce99b05700 1 -- 172.21.15.42:0/3861212381 wait complete. 2023-12-12T09:59:32.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:32 smithi173 ceph-mon[103308]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-12T09:59:32.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:32 smithi042 ceph-mon[98894]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-12T09:59:32.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:32 smithi111 ceph-mon[102565]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2023-12-12T09:59:34.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:34 smithi173 ceph-mon[103308]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-12-12T09:59:34.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:34 smithi042 ceph-mon[98894]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-12-12T09:59:34.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:34 smithi111 ceph-mon[102565]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2023-12-12T09:59:36.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:36 smithi173 ceph-mon[103308]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2023-12-12T09:59:36.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:36 smithi042 ceph-mon[98894]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2023-12-12T09:59:36.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:36 smithi111 ceph-mon[102565]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 10 op/s 2023-12-12T09:59:38.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:38 smithi173 ceph-mon[103308]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-12-12T09:59:38.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:38 smithi042 ceph-mon[98894]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-12-12T09:59:38.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:38 smithi111 ceph-mon[102565]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-12-12T09:59:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:40 smithi173 ceph-mon[103308]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-12T09:59:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:40 smithi042 ceph-mon[98894]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-12T09:59:40.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:40 smithi111 ceph-mon[102565]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-12T09:59:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:42 smithi173 ceph-mon[103308]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-12-12T09:59:42.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:42 smithi042 ceph-mon[98894]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-12-12T09:59:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:42 smithi111 ceph-mon[102565]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2023-12-12T09:59:43.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:43 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:59:43.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:43 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:59:43.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:43 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T09:59:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:44 smithi173 ceph-mon[103308]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-12T09:59:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:44 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:59:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:44 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:59:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:44 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:59:44.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:44 smithi042 ceph-mon[98894]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-12T09:59:44.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:44 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:59:44.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:44 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:59:44.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:44 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:59:44.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:44 smithi111 ceph-mon[102565]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2023-12-12T09:59:44.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:44 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T09:59:44.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:44 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T09:59:44.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:44 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T09:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.204+0000 7f331199b700 1 -- 172.21.15.42:0/3153283371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 msgr2=0x7f330c101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.204+0000 7f331199b700 1 --2- 172.21.15.42:0/3153283371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 0x7f330c101290 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009990 tx=0x7f32fc009ca0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.204+0000 7f331199b700 1 -- 172.21.15.42:0/3153283371 shutdown_connections 2023-12-12T09:59:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.204+0000 7f331199b700 1 --2- 172.21.15.42:0/3153283371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330c10df20 0x7f330c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.204+0000 7f331199b700 1 --2- 172.21.15.42:0/3153283371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 0x7f330c101290 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.204+0000 7f331199b700 1 --2- 172.21.15.42:0/3153283371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330c1017d0 0x7f330c10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.204+0000 7f331199b700 1 -- 172.21.15.42:0/3153283371 >> 172.21.15.42:0/3153283371 conn(0x7f330c077570 msgr2=0x7f330c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.204+0000 7f331199b700 1 -- 172.21.15.42:0/3153283371 shutdown_connections 2023-12-12T09:59:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.204+0000 7f331199b700 1 -- 172.21.15.42:0/3153283371 wait complete. 2023-12-12T09:59:45.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f331199b700 1 Processor -- start 2023-12-12T09:59:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f331199b700 1 -- start start 2023-12-12T09:59:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f331199b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 0x7f330c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f331199b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330c1017d0 0x7f330c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f331199b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330c10df20 0x7f330c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f331199b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c19a030 con 0x7f330c100eb0 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f331199b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c19a1a0 con 0x7f330c10df20 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f331199b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c19a360 con 0x7f330c1017d0 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f330affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 0x7f330c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f330affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 0x7f330c19f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36014/0 (socket says 172.21.15.42:36014) 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f330affd700 1 -- 172.21.15.42:0/3208929431 learned_addr learned my addr 172.21.15.42:0/3208929431 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f330a7fc700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330c1017d0 0x7f330c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.205+0000 7f330b7fe700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330c10df20 0x7f330c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f330affd700 1 -- 172.21.15.42:0/3208929431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330c1017d0 msgr2=0x7f330c19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f330affd700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330c1017d0 0x7f330c19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f330affd700 1 -- 172.21.15.42:0/3208929431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330c10df20 msgr2=0x7f330c1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f330affd700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330c10df20 0x7f330c1999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f330affd700 1 -- 172.21.15.42:0/3208929431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc0095f0 con 0x7f330c100eb0 2023-12-12T09:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f330b7fe700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330c10df20 0x7f330c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f330a7fc700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330c1017d0 0x7f330c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f330affd700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 0x7f330c19f8a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f32fc000c00 tx=0x7f32fc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f3310999700 1 -- 172.21.15.42:0/3208929431 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc01d070 con 0x7f330c100eb0 2023-12-12T09:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f331199b700 1 -- 172.21.15.42:0/3208929431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c19a5e0 con 0x7f330c100eb0 2023-12-12T09:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f3310999700 1 -- 172.21.15.42:0/3208929431 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32fc022470 con 0x7f330c100eb0 2023-12-12T09:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f3310999700 1 -- 172.21.15.42:0/3208929431 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc017650 con 0x7f330c100eb0 2023-12-12T09:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.206+0000 7f331199b700 1 -- 172.21.15.42:0/3208929431 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c1a6890 con 0x7f330c100eb0 2023-12-12T09:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.208+0000 7f3310999700 1 -- 172.21.15.42:0/3208929431 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136239+0+0 (secure 0 0 0) 0x7f32fc0225e0 con 0x7f330c100eb0 2023-12-12T09:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.209+0000 7f3310999700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f32f40a1710 0x7f32f40a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.209+0000 7f330a7fc700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f32f40a1710 0x7f32f40a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.209+0000 7f3310999700 1 -- 172.21.15.42:0/3208929431 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f32fc0ceaf0 con 0x7f330c100eb0 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.209+0000 7f330a7fc700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f32f40a1710 0x7f32f40a3bd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f330c19ab90 tx=0x7f3300009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f331db94b80 1 Processor -- start 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f331db94b80 1 -- start start 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f331db94b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 0x56202fb6c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f331db94b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 0x56202fb66580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f331db94b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb66f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f331db94b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fac1af0 con 0x56202fb66ac0 2023-12-12T09:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f331db94b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fabf410 con 0x56202fb6c660 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f331db94b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fabad00 con 0x56202fb6bd40 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb66f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb66f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36026/0 (socket says 172.21.15.42:36026) 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330b7fe700 1 -- 172.21.15.42:0/800525696 learned_addr learned my addr 172.21.15.42:0/800525696 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330a7fc700 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 0x56202fb66580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330affd700 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 0x56202fb6c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330b7fe700 1 -- 172.21.15.42:0/800525696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 msgr2=0x56202fb6c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330b7fe700 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 0x56202fb6c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330b7fe700 1 -- 172.21.15.42:0/800525696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 msgr2=0x56202fb66580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330b7fe700 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 0x56202fb66580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330b7fe700 1 -- 172.21.15.42:0/800525696 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56202f9375d0 con 0x56202fb66ac0 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330affd700 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 0x56202fb6c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330a7fc700 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 0x56202fb66580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T09:59:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.210+0000 7f330b7fe700 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb66f00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f33040093f0 tx=0x7f3304009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f25fbedd122b4b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f32f1ffb700 1 -- 172.21.15.42:0/800525696 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330400e030 con 0x56202fb66ac0 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f32f1ffb700 1 -- 172.21.15.42:0/800525696 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f330400f040 con 0x56202fb66ac0 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f32f1ffb700 1 -- 172.21.15.42:0/800525696 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304014a00 con 0x56202fb66ac0 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f331db94b80 1 -- 172.21.15.42:0/800525696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 msgr2=0x56202fb66f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f331db94b80 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb66f00 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f33040093f0 tx=0x7f3304009920 comp rx=0 tx=0).stop 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f331db94b80 1 -- 172.21.15.42:0/800525696 shutdown_connections 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f331db94b80 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb66f00 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f331db94b80 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 0x56202fb6c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f331db94b80 1 --2- 172.21.15.42:0/800525696 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 0x56202fb66580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f331db94b80 1 -- 172.21.15.42:0/800525696 >> 172.21.15.42:0/800525696 conn(0x56202fa95780 msgr2=0x56202fb60c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f331db94b80 1 -- 172.21.15.42:0/800525696 shutdown_connections 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.211+0000 7f331db94b80 1 -- 172.21.15.42:0/800525696 wait complete. 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.212+0000 7f331db94b80 1 -- start start 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.212+0000 7f331db94b80 1 Processor -- start 2023-12-12T09:59:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.212+0000 7f331db94b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.212+0000 7f331db94b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 0x56202fb6f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.212+0000 7f331db94b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 0x56202fb6ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f331db94b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fa74500 con 0x56202fb66ac0 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f331db94b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fa741a0 con 0x56202fb6c660 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f331db94b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202f9398c0 con 0x56202fb6bd40 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f331db94b80 10 client.0 _ll_register_callbacks cb 0x56202f99fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36028/0 (socket says 172.21.15.42:36028) 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330affd700 1 -- 172.21.15.42:0/4166703647 learned_addr learned my addr 172.21.15.42:0/4166703647 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330a7fc700 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 0x56202fb6f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330b7fe700 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 0x56202fb6ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330affd700 1 -- 172.21.15.42:0/4166703647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 msgr2=0x56202fb6f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330affd700 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 0x56202fb6f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330affd700 1 -- 172.21.15.42:0/4166703647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 msgr2=0x56202fb6ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330affd700 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 0x56202fb6ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330affd700 1 -- 172.21.15.42:0/4166703647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3300011c30 con 0x56202fb66ac0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f330affd700 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f32fc022890 tx=0x7f32fc022bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab45fbf4525788f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f32eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f32eaffd700 1 -- 172.21.15.42:0/4166703647 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc09bd40 con 0x56202fb66ac0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f32eaffd700 1 -- 172.21.15.42:0/4166703647 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f32fc09b5c0 con 0x56202fb66ac0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f331db94b80 1 -- 172.21.15.42:0/4166703647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56202fb59730 con 0x56202fb66ac0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f32eaffd700 1 -- 172.21.15.42:0/4166703647 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32fc09f710 con 0x56202fb66ac0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f331db94b80 10 client.15690 fetch_fsmap learned FSMap version 27 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.213+0000 7f331db94b80 1 -- 172.21.15.42:0/4166703647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f33040090d0 con 0x56202fb66ac0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.214+0000 7f32eaffd700 1 -- 172.21.15.42:0/4166703647 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56202fb59730 con 0x56202fb66ac0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.214+0000 7f32eaffd700 1 -- 172.21.15.42:0/4166703647 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f32fc0cc050 con 0x56202fb66ac0 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.214+0000 7f331db94b80 10 client.15690 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.214+0000 7f331db94b80 10 client.15690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.214+0000 7f331db94b80 20 client.15690 populate_metadata read hostname 'smithi042' 2023-12-12T09:59:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.214+0000 7f331db94b80 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56202fb7fbf0 0x56202fb81fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.214+0000 7f331db94b80 4 client.15690 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T09:59:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.214+0000 7f331db94b80 1 -- 172.21.15.42:0/4166703647 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56202fb35ee0 con 0x56202fb7fbf0 2023-12-12T09:59:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.214+0000 7f330a7fc700 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56202fb7fbf0 0x56202fb81fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.215+0000 7f32eaffd700 10 client.15690 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.215+0000 7f330a7fc700 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56202fb7fbf0 0x56202fb81fe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44db5376a31d10fb server_cookie=e12083e9b6ea570e in_seq=0 out_seq=0 2023-12-12T09:59:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.215+0000 7f32eaffd700 1 -- 172.21.15.42:0/4166703647 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56202fb35ee0 con 0x56202fb7fbf0 2023-12-12T09:59:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.215+0000 7f32eaffd700 10 client.15690 handle_command_reply: tid=0 2023-12-12T09:59:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.215+0000 7f331db94b80 1 client.15690 shutdown 2023-12-12T09:59:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.215+0000 7f331db94b80 1 -- 172.21.15.42:0/4166703647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 msgr2=0x56202fb72620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.215+0000 7f331db94b80 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72620 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f32fc022890 tx=0x7f32fc022bc0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.215+0000 7f331db94b80 1 -- 172.21.15.42:0/4166703647 shutdown_connections 2023-12-12T09:59:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.216+0000 7f331db94b80 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56202fb7fbf0 0x56202fb81fe0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.216+0000 7f331db94b80 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6c660 0x56202fb6ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.216+0000 7f331db94b80 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72620 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.216+0000 7f331db94b80 1 --2- 172.21.15.42:0/4166703647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6bd40 0x56202fb6f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.216+0000 7f331db94b80 1 -- 172.21.15.42:0/4166703647 >> 172.21.15.42:0/4166703647 conn(0x56202fa95780 msgr2=0x56202fb65570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.216+0000 7f32eaffd700 0 client.15690 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.216+0000 7f331db94b80 1 -- 172.21.15.42:0/4166703647 shutdown_connections 2023-12-12T09:59:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.216+0000 7f331db94b80 1 -- 172.21.15.42:0/4166703647 wait complete. 2023-12-12T09:59:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.216+0000 7f331db94b80 20 client.15690 trim_cache size 0 max 16384 2023-12-12T09:59:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f331db94b80 1 Processor -- start 2023-12-12T09:59:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f331db94b80 1 -- start start 2023-12-12T09:59:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f331db94b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f331db94b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6bd40 0x56202fb65dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f331db94b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6c660 0x56202fb868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f331db94b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fb7cb20 con 0x56202fb66ac0 2023-12-12T09:59:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f331db94b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fb618d0 con 0x56202fb6bd40 2023-12-12T09:59:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f331db94b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fb66900 con 0x56202fb6c660 2023-12-12T09:59:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f330affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f330affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36036/0 (socket says 172.21.15.42:36036) 2023-12-12T09:59:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.231+0000 7f330affd700 1 -- 172.21.15.42:0/2576767387 learned_addr learned my addr 172.21.15.42:0/2576767387 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f330a7fc700 1 --2- 172.21.15.42:0/2576767387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6bd40 0x56202fb65dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f330b7fe700 1 --2- 172.21.15.42:0/2576767387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6c660 0x56202fb868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f330affd700 1 -- 172.21.15.42:0/2576767387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6c660 msgr2=0x56202fb868d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f330affd700 1 --2- 172.21.15.42:0/2576767387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6c660 0x56202fb868d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f330affd700 1 -- 172.21.15.42:0/2576767387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6bd40 msgr2=0x56202fb65dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f330affd700 1 --2- 172.21.15.42:0/2576767387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6bd40 0x56202fb65dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f330affd700 1 -- 172.21.15.42:0/2576767387 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56202f8e3d80 con 0x56202fb66ac0 2023-12-12T09:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f330affd700 1 --2- 172.21.15.42:0/2576767387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f32fc098030 tx=0x7f32fc022c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf5a9146a8a1469 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f32f27fc700 1 -- 172.21.15.42:0/2576767387 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc084ea0 con 0x56202fb66ac0 2023-12-12T09:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f32f27fc700 1 -- 172.21.15.42:0/2576767387 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32fc097900 con 0x56202fb66ac0 2023-12-12T09:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.232+0000 7f32f27fc700 1 -- 172.21.15.42:0/2576767387 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc0cc050 con 0x56202fb66ac0 2023-12-12T09:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 -- 172.21.15.42:0/2576767387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 msgr2=0x56202fb72350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 --2- 172.21.15.42:0/2576767387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72350 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f32fc098030 tx=0x7f32fc022c80 comp rx=0 tx=0).stop 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 -- 172.21.15.42:0/2576767387 shutdown_connections 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 --2- 172.21.15.42:0/2576767387 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6c660 0x56202fb868d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 --2- 172.21.15.42:0/2576767387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb66ac0 0x56202fb72350 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 --2- 172.21.15.42:0/2576767387 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb6bd40 0x56202fb65dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 -- 172.21.15.42:0/2576767387 >> 172.21.15.42:0/2576767387 conn(0x56202fa95780 msgr2=0x56202fb65570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 -- 172.21.15.42:0/2576767387 shutdown_connections 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 -- 172.21.15.42:0/2576767387 wait complete. 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.233+0000 7f331db94b80 1 -- start start 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f331db94b80 1 Processor -- start 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f331db94b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb66ac0 0x56202fb77270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f331db94b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb6bd40 0x56202fb6a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f331db94b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6c660 0x56202fb6af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f331db94b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fb6b500 con 0x56202fb6bd40 2023-12-12T09:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f331db94b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fb6b640 con 0x56202fb66ac0 2023-12-12T09:59:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f331db94b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202fb6b800 con 0x56202fb6c660 2023-12-12T09:59:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f331db94b80 10 client.0 _ll_register_callbacks cb 0x56202f99fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T09:59:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb6bd40 0x56202fb6a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330a7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb6bd40 0x56202fb6a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36044/0 (socket says 172.21.15.42:36044) 2023-12-12T09:59:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330a7fc700 1 -- 172.21.15.42:0/830113281 learned_addr learned my addr 172.21.15.42:0/830113281 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T09:59:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330affd700 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb66ac0 0x56202fb77270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330a7fc700 1 -- 172.21.15.42:0/830113281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6c660 msgr2=0x56202fb6af30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-12T09:59:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330a7fc700 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6c660 0x56202fb6af30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330a7fc700 1 -- 172.21.15.42:0/830113281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb66ac0 msgr2=0x56202fb77270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T09:59:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330a7fc700 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb66ac0 0x56202fb77270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330a7fc700 1 -- 172.21.15.42:0/830113281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33040090d0 con 0x56202fb6bd40 2023-12-12T09:59:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.234+0000 7f330affd700 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb66ac0 0x56202fb77270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T09:59:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.235+0000 7f330a7fc700 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb6bd40 0x56202fb6a9f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3300009810 tx=0x7f33000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82547d5b68941e88 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T09:59:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.235+0000 7f32eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T09:59:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.235+0000 7f32eaffd700 1 -- 172.21.15.42:0/830113281 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300024040 con 0x56202fb6bd40 2023-12-12T09:59:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.235+0000 7f32eaffd700 1 -- 172.21.15.42:0/830113281 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3300018070 con 0x56202fb6bd40 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.235+0000 7f331db94b80 1 -- 172.21.15.42:0/830113281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f330400f040 con 0x56202fb6bd40 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.235+0000 7f32eaffd700 1 -- 172.21.15.42:0/830113281 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33000297b0 con 0x56202fb6bd40 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.235+0000 7f331db94b80 10 client.15696 fetch_fsmap learned FSMap version 27 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.235+0000 7f331db94b80 1 -- 172.21.15.42:0/830113281 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56202f8ccc00 con 0x56202fb6bd40 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.235+0000 7f32eaffd700 1 -- 172.21.15.42:0/830113281 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f330400f040 con 0x56202fb6bd40 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f32eaffd700 1 -- 172.21.15.42:0/830113281 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f330001f480 con 0x56202fb6bd40 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f331db94b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f331db94b80 10 client.15696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f331db94b80 20 client.15696 populate_metadata read hostname 'smithi042' 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f331db94b80 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56202fbd6770 0x56202fbd8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f331db94b80 4 client.15696 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f331db94b80 1 -- 172.21.15.42:0/830113281 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56202fa80f70 con 0x56202fbd6770 2023-12-12T09:59:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f330affd700 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56202fbd6770 0x56202fbd8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T09:59:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f32eaffd700 10 client.15696 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.236+0000 7f330affd700 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56202fbd6770 0x56202fbd8b90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1dbf24cd693e2d4 server_cookie=8728d904f8dea0a1 in_seq=0 out_seq=0 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.240+0000 7f32eaffd700 1 -- 172.21.15.42:0/830113281 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56202fa80f70 con 0x56202fbd6770 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.240+0000 7f32eaffd700 10 client.15696 handle_command_reply: tid=0 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.240+0000 7f331db94b80 1 client.15696 shutdown 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.240+0000 7f331db94b80 1 -- 172.21.15.42:0/830113281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb6bd40 msgr2=0x56202fb6a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.240+0000 7f331db94b80 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb6bd40 0x56202fb6a9f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3300009810 tx=0x7f33000047c0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331db94b80 1 -- 172.21.15.42:0/830113281 shutdown_connections 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331db94b80 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x56202fbd6770 0x56202fbd8b90 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331db94b80 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x56202fb66ac0 0x56202fb77270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331db94b80 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x56202fb6c660 0x56202fb6af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f32eaffd700 0 client.15696 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331db94b80 1 --2- 172.21.15.42:0/830113281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56202fb6bd40 0x56202fb6a9f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331db94b80 1 -- 172.21.15.42:0/830113281 >> 172.21.15.42:0/830113281 conn(0x56202fa95780 msgr2=0x56202fb609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331db94b80 1 -- 172.21.15.42:0/830113281 shutdown_connections 2023-12-12T09:59:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331db94b80 1 -- 172.21.15.42:0/830113281 wait complete. 2023-12-12T09:59:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331db94b80 20 client.15696 trim_cache size 0 max 16384 2023-12-12T09:59:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T09:59:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T09:59:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T09:59:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 -- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f32f40a1710 msgr2=0x7f32f40a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f32f40a1710 0x7f32f40a3bd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f330c19ab90 tx=0x7f3300009260 comp rx=0 tx=0).stop 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 -- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 msgr2=0x7f330c19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 0x7f330c19f8a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f32fc000c00 tx=0x7f32fc005e80 comp rx=0 tx=0).stop 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 -- 172.21.15.42:0/3208929431 shutdown_connections 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f32f40a1710 0x7f32f40a3bd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f330c10df20 0x7f330c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f330c100eb0 0x7f330c19f8a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 --2- 172.21.15.42:0/3208929431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f330c1017d0 0x7f330c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.241+0000 7f331199b700 1 -- 172.21.15.42:0/3208929431 >> 172.21.15.42:0/3208929431 conn(0x7f330c077570 msgr2=0x7f330c10f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T09:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.242+0000 7f331199b700 1 -- 172.21.15.42:0/3208929431 shutdown_connections 2023-12-12T09:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T09:59:45.242+0000 7f331199b700 1 -- 172.21.15.42:0/3208929431 wait complete. 2023-12-12T09:59:45.633 INFO:tasks.workunit.client.0.smithi042.stderr:Updating files: 95% (11618/12157) Updating files: 96% (11671/12157) Updating files: 97% (11793/12157) Updating files: 98% (11914/12157) Updating files: 99% (12036/12157) Updating files: 100% (12157/12157) Updating files: 100% (12157/12157), done. 2023-12-12T09:59:46.150 INFO:tasks.workunit.client.0.smithi042.stderr:Note: switching to 'f1fbf3c583d03e19f5e58eb618e78d8e10d075f9'. 2023-12-12T09:59:46.150 INFO:tasks.workunit.client.0.smithi042.stderr: 2023-12-12T09:59:46.150 INFO:tasks.workunit.client.0.smithi042.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-12-12T09:59:46.150 INFO:tasks.workunit.client.0.smithi042.stderr:changes and commit them, and you can discard any commits you make in this 2023-12-12T09:59:46.150 INFO:tasks.workunit.client.0.smithi042.stderr:state without impacting any branches by switching back to a branch. 2023-12-12T09:59:46.150 INFO:tasks.workunit.client.0.smithi042.stderr: 2023-12-12T09:59:46.150 INFO:tasks.workunit.client.0.smithi042.stderr:If you want to create a new branch to retain commits you create, you may 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr:do so (now or later) by using -c with the switch command. Example: 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr: 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr: git switch -c 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr: 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr:Or undo this operation with: 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr: 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr: git switch - 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr: 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr: 2023-12-12T09:59:46.151 INFO:tasks.workunit.client.0.smithi042.stderr:HEAD is now at f1fbf3c583d qa: add split/merge dirfrag tests for scrub 2023-12-12T09:59:46.152 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-12-12T09:59:46.218 INFO:tasks.workunit.client.0.smithi042.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-12-12T09:59:46.220 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-12T09:59:46.220 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-12-12T09:59:46.786 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-12-12T09:59:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:46 smithi173 ceph-mon[103308]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-12T09:59:46.838 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-12-12T09:59:46.878 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-12T09:59:46.879 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-12T09:59:46.879 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-12-12T09:59:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:46 smithi042 ceph-mon[98894]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-12T09:59:46.917 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-12T09:59:46.920 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T09:59:46.920 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-12-12T09:59:46.979 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2023-12-12T09:59:46.981 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2023-12-12T09:59:46.981 DEBUG:teuthology.orchestra.run.smithi042:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2023-12-12T09:59:46.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:46 smithi111 ceph-mon[102565]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-12T09:59:47.043 INFO:tasks.workunit.client.0.smithi042.stderr:+ set -e 2023-12-12T09:59:47.043 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir -p testdir 2023-12-12T09:59:47.045 INFO:tasks.workunit.client.0.smithi042.stderr:+ cd testdir 2023-12-12T09:59:47.045 INFO:tasks.workunit.client.0.smithi042.stderr:+ set +e 2023-12-12T09:59:47.045 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfacl -d -m u:nobody:rw . 2023-12-12T09:59:47.056 INFO:tasks.workunit.client.0.smithi042.stderr:+ test 0 '!=' 0 2023-12-12T09:59:47.056 INFO:tasks.workunit.client.0.smithi042.stderr:+ set -e 2023-12-12T09:59:47.056 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=0 2023-12-12T09:59:47.057 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 0 -lt 100 ']' 2023-12-12T09:59:47.057 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 0 + 1 2023-12-12T09:59:47.057 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=1 2023-12-12T09:59:47.057 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:47.059 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:47.059 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:47.064 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:47.064 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:47.064 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:47.819 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:47.819 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:48.001 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:48.001 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:48.012 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:48.012 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:48.012 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 1 -lt 100 ']' 2023-12-12T09:59:48.012 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 1 + 1 2023-12-12T09:59:48.036 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=2 2023-12-12T09:59:48.036 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:48.073 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:48.073 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:48.073 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:48.074 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:48.074 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:48.468 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:48.469 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:48.592 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:48.592 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:48.604 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:48.604 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:48.604 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -lt 100 ']' 2023-12-12T09:59:48.604 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 2 + 1 2023-12-12T09:59:48.624 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=3 2023-12-12T09:59:48.624 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:48.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:48 smithi042 ceph-mon[98894]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-12T09:59:48.674 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:48.675 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:48.675 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:48.675 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:48.676 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:48 smithi173 ceph-mon[103308]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-12T09:59:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:48 smithi111 ceph-mon[102565]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2023-12-12T09:59:49.051 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:49.052 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:49.167 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:49.168 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:49.180 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:49.180 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:49.180 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 3 -lt 100 ']' 2023-12-12T09:59:49.180 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 3 + 1 2023-12-12T09:59:49.227 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=4 2023-12-12T09:59:49.227 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:49.274 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:49.274 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:49.275 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:49.275 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:49.275 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:49.810 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:49.810 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:49.934 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:49.934 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:49.954 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:49.954 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:49.954 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 4 -lt 100 ']' 2023-12-12T09:59:49.954 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 4 + 1 2023-12-12T09:59:49.982 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=5 2023-12-12T09:59:49.982 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:50.016 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:50.016 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:50.017 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:50.017 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:50.017 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:50.351 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:50.352 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:50.449 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:50.449 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:50.463 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:50.463 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:50.464 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 5 -lt 100 ']' 2023-12-12T09:59:50.464 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 5 + 1 2023-12-12T09:59:50.490 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=6 2023-12-12T09:59:50.491 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:50.509 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:50.509 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:50.510 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:50.510 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:50.510 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:50.818 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:50.818 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:50.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:50 smithi173 ceph-mon[103308]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-12T09:59:50.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:50 smithi042 ceph-mon[98894]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-12T09:59:50.917 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:50.917 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:50.928 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:50.929 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:50.929 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 6 -lt 100 ']' 2023-12-12T09:59:50.929 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 6 + 1 2023-12-12T09:59:50.949 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=7 2023-12-12T09:59:50.949 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:50.977 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:50.978 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:50.978 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:50.978 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:50.978 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:50 smithi111 ceph-mon[102565]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.7 KiB/s wr, 13 op/s 2023-12-12T09:59:51.293 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:51.293 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:51.460 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:51.460 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:51.484 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:51.484 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:51.484 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 7 -lt 100 ']' 2023-12-12T09:59:51.485 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 7 + 1 2023-12-12T09:59:51.540 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=8 2023-12-12T09:59:51.540 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:51.567 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:51.568 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:51.569 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:51.569 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:51.569 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:51.926 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:51.926 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:52.080 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:52.080 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:52.103 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:52.103 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:52.103 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 8 -lt 100 ']' 2023-12-12T09:59:52.103 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 8 + 1 2023-12-12T09:59:52.134 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=9 2023-12-12T09:59:52.134 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:52.150 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:52.150 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:52.151 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:52.151 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:52.151 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:52.501 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:52.501 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:52.625 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:52.625 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:52.637 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:52.637 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:52.637 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 9 -lt 100 ']' 2023-12-12T09:59:52.637 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 9 + 1 2023-12-12T09:59:52.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:52 smithi042 ceph-mon[98894]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-12-12T09:59:52.657 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=10 2023-12-12T09:59:52.657 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:52.675 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:52.675 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:52.676 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:52.676 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:52.676 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:52.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:52 smithi173 ceph-mon[103308]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-12-12T09:59:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:52 smithi111 ceph-mon[102565]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-12-12T09:59:53.026 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:53.026 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:53.150 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:53.150 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:53.161 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:53.162 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:53.162 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 10 -lt 100 ']' 2023-12-12T09:59:53.162 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 10 + 1 2023-12-12T09:59:53.182 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=11 2023-12-12T09:59:53.182 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:53.199 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:53.200 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:53.200 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:53.200 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:53.200 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:53.586 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:53.586 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:53.776 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:53.776 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:53.792 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:53.792 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:53.792 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 11 -lt 100 ']' 2023-12-12T09:59:53.792 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 11 + 1 2023-12-12T09:59:53.815 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=12 2023-12-12T09:59:53.816 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:53.833 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:53.834 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:53.835 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:53.835 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:53.835 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:54.184 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:54.185 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:54.292 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:54.292 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:54.303 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:54.304 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:54.304 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 12 -lt 100 ']' 2023-12-12T09:59:54.304 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 12 + 1 2023-12-12T09:59:54.331 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=13 2023-12-12T09:59:54.332 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:54.350 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:54.351 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:54.351 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:54.351 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:54.351 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:54.701 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:54.701 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:54.808 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:54.808 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:54.821 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:54.822 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:54.822 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 13 -lt 100 ']' 2023-12-12T09:59:54.822 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 13 + 1 2023-12-12T09:59:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:54 smithi173 ceph-mon[103308]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2023-12-12T09:59:54.848 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=14 2023-12-12T09:59:54.848 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:54.869 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:54.869 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:54.870 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:54.870 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:54.870 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:54 smithi042 ceph-mon[98894]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2023-12-12T09:59:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:54 smithi111 ceph-mon[102565]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 9.8 KiB/s wr, 14 op/s 2023-12-12T09:59:55.218 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:55.218 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:55.358 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:55.358 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:55.370 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:55.370 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:55.370 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 14 -lt 100 ']' 2023-12-12T09:59:55.370 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 14 + 1 2023-12-12T09:59:55.398 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=15 2023-12-12T09:59:55.399 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:55.424 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:55.424 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:55.425 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:55.425 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:55.425 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:55.834 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:55.835 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:55.933 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:55.933 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:55.958 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:55.958 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:55.958 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 15 -lt 100 ']' 2023-12-12T09:59:55.958 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 15 + 1 2023-12-12T09:59:55.982 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=16 2023-12-12T09:59:55.982 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:55.999 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:56.000 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:56.000 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:56.000 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:56.001 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:56.351 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:56.351 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:56.458 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:56.458 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:56.469 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:56.469 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:56.469 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 16 -lt 100 ']' 2023-12-12T09:59:56.469 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 16 + 1 2023-12-12T09:59:56.489 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=17 2023-12-12T09:59:56.490 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:56.520 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:56.520 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:56.521 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:56.521 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:56.521 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:56.819 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:56 smithi042 ceph-mon[98894]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2023-12-12T09:59:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:56 smithi173 ceph-mon[103308]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2023-12-12T09:59:56.859 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:56.859 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:56.974 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:56.974 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:56.995 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:56.995 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:56.995 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 17 -lt 100 ']' 2023-12-12T09:59:56.995 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 17 + 1 2023-12-12T09:59:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:56 smithi111 ceph-mon[102565]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2023-12-12T09:59:57.023 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=18 2023-12-12T09:59:57.024 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:57.042 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:57.042 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:57.042 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:57.043 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:57.043 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:57.484 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:57.484 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:57.608 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:57.608 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:57.619 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:57.619 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:57.619 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 18 -lt 100 ']' 2023-12-12T09:59:57.620 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 18 + 1 2023-12-12T09:59:57.648 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=19 2023-12-12T09:59:57.648 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:57.667 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:57.667 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:57.668 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:57.668 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:57.668 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:58.075 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:58.076 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:58.191 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:58.191 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:58.203 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:58.203 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:58.204 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 19 -lt 100 ']' 2023-12-12T09:59:58.204 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 19 + 1 2023-12-12T09:59:58.231 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=20 2023-12-12T09:59:58.231 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:58.266 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:58.266 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:58.266 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:58.266 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:58.267 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:58.684 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:58.684 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 09:59:58 smithi173 ceph-mon[103308]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2023-12-12T09:59:58.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 09:59:58 smithi042 ceph-mon[98894]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2023-12-12T09:59:58.916 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:58.916 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:58.937 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:58.937 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:58.937 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 20 -lt 100 ']' 2023-12-12T09:59:58.937 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 20 + 1 2023-12-12T09:59:58.984 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=21 2023-12-12T09:59:58.984 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T09:59:58.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 09:59:58 smithi111 ceph-mon[102565]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.8 KiB/s wr, 9 op/s 2023-12-12T09:59:59.013 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:59.014 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:59.015 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T09:59:59.015 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T09:59:59.015 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T09:59:59.742 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T09:59:59.742 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T09:59:59.924 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T09:59:59.924 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T09:59:59.945 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:59.945 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T09:59:59.945 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 21 -lt 100 ']' 2023-12-12T09:59:59.945 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 21 + 1 2023-12-12T10:00:00.006 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=22 2023-12-12T10:00:00.006 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:00.065 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:00.065 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:00.066 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:00.066 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:00.066 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 -- 172.21.15.42:0/3719453398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d410bad0 msgr2=0x7f34d410df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 --2- 172.21.15.42:0/3719453398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d410bad0 0x7f34d410df70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00b0a0 tx=0x7f34cc00b3b0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 -- 172.21.15.42:0/3719453398 shutdown_connections 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 --2- 172.21.15.42:0/3719453398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d410bad0 0x7f34d410df70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 --2- 172.21.15.42:0/3719453398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34d4102700 0x7f34d4102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 --2- 172.21.15.42:0/3719453398 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d4103020 0x7f34d410b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 -- 172.21.15.42:0/3719453398 >> 172.21.15.42:0/3719453398 conn(0x7f34d40fbea0 msgr2=0x7f34d40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 -- 172.21.15.42:0/3719453398 shutdown_connections 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 -- 172.21.15.42:0/3719453398 wait complete. 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.338+0000 7f34d9575700 1 Processor -- start 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d9575700 1 -- start start 2023-12-12T10:00:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d9575700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d4102700 0x7f34d419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d9575700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34d4103020 0x7f34d419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d9575700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d410bad0 0x7f34d41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d9575700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d4198040 con 0x7f34d4102700 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d9575700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d41981b0 con 0x7f34d410bad0 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d9575700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d4198370 con 0x7f34d4103020 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d4102700 0x7f34d419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d4102700 0x7f34d419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36060/0 (socket says 172.21.15.42:36060) 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d2ffd700 1 -- 172.21.15.42:0/4023402134 learned_addr learned my addr 172.21.15.42:0/4023402134 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d37fe700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d410bad0 0x7f34d41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d27fc700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34d4103020 0x7f34d419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d2ffd700 1 -- 172.21.15.42:0/4023402134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34d4103020 msgr2=0x7f34d419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34d4103020 0x7f34d419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d2ffd700 1 -- 172.21.15.42:0/4023402134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d410bad0 msgr2=0x7f34d41979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d410bad0 0x7f34d41979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d2ffd700 1 -- 172.21.15.42:0/4023402134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc009d00 con 0x7f34d4102700 2023-12-12T10:00:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.339+0000 7f34d27fc700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34d4103020 0x7f34d419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:00:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.340+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d4102700 0x7f34d419d860 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f34c800b530 tx=0x7f34c800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.340+0000 7f34bbfff700 1 -- 172.21.15.42:0/4023402134 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8011830 con 0x7f34d4102700 2023-12-12T10:00:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.340+0000 7f34bbfff700 1 -- 172.21.15.42:0/4023402134 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34c8011e50 con 0x7f34d4102700 2023-12-12T10:00:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.340+0000 7f34d9575700 1 -- 172.21.15.42:0/4023402134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d4198650 con 0x7f34d4102700 2023-12-12T10:00:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.340+0000 7f34bbfff700 1 -- 172.21.15.42:0/4023402134 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c800f2e0 con 0x7f34d4102700 2023-12-12T10:00:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.340+0000 7f34d9575700 1 -- 172.21.15.42:0/4023402134 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d41a4850 con 0x7f34d4102700 2023-12-12T10:00:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.342+0000 7f34bbfff700 1 -- 172.21.15.42:0/4023402134 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 24) v1 ==== 136239+0+0 (secure 0 0 0) 0x7f34c8010380 con 0x7f34d4102700 2023-12-12T10:00:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.343+0000 7f34bbfff700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34bc0a1710 0x7f34bc0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.343+0000 7f34d27fc700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34bc0a1710 0x7f34bc0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.343+0000 7f34bbfff700 1 -- 172.21.15.42:0/4023402134 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f34c80cd0d0 con 0x7f34d4102700 2023-12-12T10:00:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.343+0000 7f34d27fc700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34bc0a1710 0x7f34bc0a3bd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34c4005fd0 tx=0x7f34c4005ee0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.392+0000 7f34e576eb80 1 Processor -- start 2023-12-12T10:00:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.392+0000 7f34e576eb80 1 -- start start 2023-12-12T10:00:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86c9f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86c99580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86c99f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34e576eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86bf4af0 con 0x55ba86c9ed40 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34e576eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86bf2410 con 0x55ba86c9f660 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34e576eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86bedd00 con 0x55ba86c99ac0 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86c9f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86c9f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36068/0 (socket says 172.21.15.42:36068) 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d2ffd700 1 -- 172.21.15.42:0/3354766170 learned_addr learned my addr 172.21.15.42:0/3354766170 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d27fc700 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86c99580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d37fe700 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86c99f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d2ffd700 1 -- 172.21.15.42:0/3354766170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 msgr2=0x55ba86c99f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86c99f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d2ffd700 1 -- 172.21.15.42:0/3354766170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 msgr2=0x55ba86c99580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86c99580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d2ffd700 1 -- 172.21.15.42:0/3354766170 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba86a6a5d0 con 0x55ba86c9ed40 2023-12-12T10:00:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d37fe700 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86c99f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:00:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d27fc700 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86c99580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:00:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.393+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86c9f120 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f34c80999f0 tx=0x7f34c8004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2df2d5948cd550 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34b97fa700 1 -- 172.21.15.42:0/3354766170 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c809de40 con 0x55ba86c9ed40 2023-12-12T10:00:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34b97fa700 1 -- 172.21.15.42:0/3354766170 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34c80a2080 con 0x55ba86c9ed40 2023-12-12T10:00:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34b97fa700 1 -- 172.21.15.42:0/3354766170 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c80a1e30 con 0x55ba86c9ed40 2023-12-12T10:00:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34e576eb80 1 -- 172.21.15.42:0/3354766170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 msgr2=0x55ba86c9f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:00.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86c9f120 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f34c80999f0 tx=0x7f34c8004860 comp rx=0 tx=0).stop 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34e576eb80 1 -- 172.21.15.42:0/3354766170 shutdown_connections 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86c99f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86c9f120 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3354766170 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86c99580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34e576eb80 1 -- 172.21.15.42:0/3354766170 >> 172.21.15.42:0/3354766170 conn(0x55ba86bc8780 msgr2=0x55ba86c93c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34e576eb80 1 -- 172.21.15.42:0/3354766170 shutdown_connections 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.394+0000 7f34e576eb80 1 -- 172.21.15.42:0/3354766170 wait complete. 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34e576eb80 1 -- start start 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34e576eb80 1 Processor -- start 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86ca2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86ca2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86ca3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34e576eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86ba7500 con 0x55ba86c9ed40 2023-12-12T10:00:00.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34e576eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86ba71a0 con 0x55ba86c9f660 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34e576eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86a6c8c0 con 0x55ba86c99ac0 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34e576eb80 10 client.0 _ll_register_callbacks cb 0x55ba86ad2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.395+0000 7f34d27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86ca2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86ca2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36076/0 (socket says 172.21.15.42:36076) 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d27fc700 1 -- 172.21.15.42:0/1464932725 learned_addr learned my addr 172.21.15.42:0/1464932725 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d37fe700 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86ca3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86ca2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d27fc700 1 -- 172.21.15.42:0/1464932725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 msgr2=0x55ba86ca2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d27fc700 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86ca2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d27fc700 1 -- 172.21.15.42:0/1464932725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 msgr2=0x55ba86ca3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:00.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d27fc700 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86ca3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d27fc700 1 -- 172.21.15.42:0/1464932725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d41a4850 con 0x55ba86c9ed40 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86ca2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34d27fc700 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86ca2ed0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f34c40117a0 tx=0x7f34c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01dfe8021eac906 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34b6ff5700 1 -- 172.21.15.42:0/1464932725 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c401f830 con 0x55ba86c9ed40 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34b6ff5700 1 -- 172.21.15.42:0/1464932725 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f34c401f990 con 0x55ba86c9ed40 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34e576eb80 1 -- 172.21.15.42:0/1464932725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba86c8c730 con 0x55ba86c9ed40 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34b6ff5700 1 -- 172.21.15.42:0/1464932725 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34c401fb50 con 0x55ba86c9ed40 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34e576eb80 10 client.15708 fetch_fsmap learned FSMap version 27 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.396+0000 7f34e576eb80 1 -- 172.21.15.42:0/1464932725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ba86a16d80 con 0x55ba86c9ed40 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.397+0000 7f34b6ff5700 1 -- 172.21.15.42:0/1464932725 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba86c8c730 con 0x55ba86c9ed40 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.397+0000 7f34b6ff5700 1 -- 172.21.15.42:0/1464932725 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f34c40273b0 con 0x55ba86c9ed40 2023-12-12T10:00:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.397+0000 7f34e576eb80 10 client.15708 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.397+0000 7f34e576eb80 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.397+0000 7f34e576eb80 20 client.15708 populate_metadata read hostname 'smithi042' 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.397+0000 7f34e576eb80 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ba86cb30c0 0x55ba86cb54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.397+0000 7f34e576eb80 4 client.15708 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.397+0000 7f34e576eb80 1 -- 172.21.15.42:0/1464932725 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba86c68ee0 con 0x55ba86cb30c0 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.397+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ba86cb30c0 0x55ba86cb54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34b6ff5700 10 client.15708 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ba86cb30c0 0x55ba86cb54e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c4c0041d21ba0d2 server_cookie=3ea5c3e46f29511c in_seq=0 out_seq=0 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34b6ff5700 1 -- 172.21.15.42:0/1464932725 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba86c68ee0 con 0x55ba86cb30c0 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34b6ff5700 10 client.15708 handle_command_reply: tid=0 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34e576eb80 1 client.15708 shutdown 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34e576eb80 1 -- 172.21.15.42:0/1464932725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 msgr2=0x55ba86ca2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34e576eb80 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86ca2ed0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f34c40117a0 tx=0x7f34c40047c0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34e576eb80 1 -- 172.21.15.42:0/1464932725 shutdown_connections 2023-12-12T10:00:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34e576eb80 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ba86cb30c0 0x55ba86cb54e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34e576eb80 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86ca3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34e576eb80 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c99ac0 0x55ba86ca2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34e576eb80 1 --2- 172.21.15.42:0/1464932725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86ca2ed0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34b6ff5700 0 client.15708 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.398+0000 7f34e576eb80 1 -- 172.21.15.42:0/1464932725 >> 172.21.15.42:0/1464932725 conn(0x55ba86bc8780 msgr2=0x55ba86c98570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.399+0000 7f34e576eb80 1 -- 172.21.15.42:0/1464932725 shutdown_connections 2023-12-12T10:00:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.399+0000 7f34e576eb80 1 -- 172.21.15.42:0/1464932725 wait complete. 2023-12-12T10:00:00.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.399+0000 7f34e576eb80 20 client.15708 trim_cache size 0 max 16384 2023-12-12T10:00:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34e576eb80 1 Processor -- start 2023-12-12T10:00:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34e576eb80 1 -- start start 2023-12-12T10:00:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c99ac0 0x55ba86cb54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86caa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9f660 0x55ba86caa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34e576eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86cb0770 con 0x55ba86c9ed40 2023-12-12T10:00:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34e576eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86c8c700 con 0x55ba86c99ac0 2023-12-12T10:00:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34e576eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86cafbf0 con 0x55ba86c9f660 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86caa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86caa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36084/0 (socket says 172.21.15.42:36084) 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d27fc700 1 -- 172.21.15.42:0/860833945 learned_addr learned my addr 172.21.15.42:0/860833945 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/860833945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c99ac0 0x55ba86cb54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d27fc700 1 -- 172.21.15.42:0/860833945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9f660 msgr2=0x55ba86caa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d37fe700 1 --2- 172.21.15.42:0/860833945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9f660 0x55ba86caa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d27fc700 1 --2- 172.21.15.42:0/860833945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9f660 0x55ba86caa580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d27fc700 1 -- 172.21.15.42:0/860833945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c99ac0 msgr2=0x55ba86cb54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d27fc700 1 --2- 172.21.15.42:0/860833945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c99ac0 0x55ba86cb54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d27fc700 1 -- 172.21.15.42:0/860833945 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba869ffc00 con 0x55ba86c9ed40 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.414+0000 7f34d27fc700 1 --2- 172.21.15.42:0/860833945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86caa040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f34c4011770 tx=0x7f34c400e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a460176dafc683f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34b9ffb700 1 -- 172.21.15.42:0/860833945 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4037070 con 0x55ba86c9ed40 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34b9ffb700 1 -- 172.21.15.42:0/860833945 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34c40044e0 con 0x55ba86c9ed40 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34b9ffb700 1 -- 172.21.15.42:0/860833945 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4004830 con 0x55ba86c9ed40 2023-12-12T10:00:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34e576eb80 1 -- 172.21.15.42:0/860833945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 msgr2=0x55ba86caa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34e576eb80 1 --2- 172.21.15.42:0/860833945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86caa040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f34c4011770 tx=0x7f34c400e680 comp rx=0 tx=0).stop 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34e576eb80 1 -- 172.21.15.42:0/860833945 shutdown_connections 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34e576eb80 1 --2- 172.21.15.42:0/860833945 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c99ac0 0x55ba86cb54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34e576eb80 1 --2- 172.21.15.42:0/860833945 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9f660 0x55ba86caa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34e576eb80 1 --2- 172.21.15.42:0/860833945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c9ed40 0x55ba86caa040 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34e576eb80 1 -- 172.21.15.42:0/860833945 >> 172.21.15.42:0/860833945 conn(0x55ba86bc8780 msgr2=0x55ba86ca1280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34e576eb80 1 -- 172.21.15.42:0/860833945 shutdown_connections 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.415+0000 7f34e576eb80 1 -- 172.21.15.42:0/860833945 wait complete. 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34e576eb80 1 -- start start 2023-12-12T10:00:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34e576eb80 1 Processor -- start 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c99ac0 0x55ba86ca8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9ed40 0x55ba86ca8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34e576eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86ca8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34e576eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86d08650 con 0x55ba86c99ac0 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34e576eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86d087c0 con 0x55ba86c9f660 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34e576eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba86d08980 con 0x55ba86c9ed40 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34e576eb80 10 client.0 _ll_register_callbacks cb 0x55ba86ad2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c99ac0 0x55ba86ca8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c99ac0 0x55ba86ca8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36088/0 (socket says 172.21.15.42:36088) 2023-12-12T10:00:00.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d2ffd700 1 -- 172.21.15.42:0/3332176926 learned_addr learned my addr 172.21.15.42:0/3332176926 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d37fe700 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86ca8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d27fc700 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9ed40 0x55ba86ca8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d2ffd700 1 -- 172.21.15.42:0/3332176926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9ed40 msgr2=0x55ba86ca8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9ed40 0x55ba86ca8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d2ffd700 1 -- 172.21.15.42:0/3332176926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 msgr2=0x55ba86ca8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86ca8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d2ffd700 1 -- 172.21.15.42:0/3332176926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba86a16d80 con 0x55ba86c99ac0 2023-12-12T10:00:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d27fc700 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9ed40 0x55ba86ca8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:00:00.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34d2ffd700 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c99ac0 0x55ba86ca8020 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f34c809e000 tx=0x7f34c8099830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f56de02c2b87a23 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.416+0000 7f34b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34b6ff5700 1 -- 172.21.15.42:0/3332176926 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c80993e0 con 0x55ba86c99ac0 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34b6ff5700 1 -- 172.21.15.42:0/3332176926 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f34c80a1bf0 con 0x55ba86c99ac0 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34e576eb80 1 -- 172.21.15.42:0/3332176926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34c80a2080 con 0x55ba86c99ac0 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34b6ff5700 1 -- 172.21.15.42:0/3332176926 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34c8098110 con 0x55ba86c99ac0 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34e576eb80 10 client.15714 fetch_fsmap learned FSMap version 27 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34e576eb80 1 -- 172.21.15.42:0/3332176926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ba86a3d1f0 con 0x55ba86c99ac0 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34b6ff5700 1 -- 172.21.15.42:0/3332176926 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34c80a2080 con 0x55ba86c99ac0 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34b6ff5700 1 -- 172.21.15.42:0/3332176926 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f34c8095ae0 con 0x55ba86c99ac0 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34e576eb80 10 client.15714 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:00:00.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34e576eb80 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:00:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34e576eb80 20 client.15714 populate_metadata read hostname 'smithi042' 2023-12-12T10:00:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ba86d09f40 0x55ba86d0c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34e576eb80 4 client.15714 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:00:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34e576eb80 1 -- 172.21.15.42:0/3332176926 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ba86bb3f70 con 0x55ba86d09f40 2023-12-12T10:00:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.417+0000 7f34d27fc700 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ba86d09f40 0x55ba86d0c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.418+0000 7f34d27fc700 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ba86d09f40 0x55ba86d0c330 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=338a92234ed720ee server_cookie=c285f806c8a85d66 in_seq=0 out_seq=0 2023-12-12T10:00:00.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.418+0000 7f34b6ff5700 10 client.15714 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.443+0000 7f34b6ff5700 1 -- 172.21.15.42:0/3332176926 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ba86bb3f70 con 0x55ba86d09f40 2023-12-12T10:00:00.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.443+0000 7f34b6ff5700 10 client.15714 handle_command_reply: tid=0 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.443+0000 7f34e576eb80 1 client.15714 shutdown 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.443+0000 7f34e576eb80 1 -- 172.21.15.42:0/3332176926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c99ac0 msgr2=0x55ba86ca8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.443+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c99ac0 0x55ba86ca8020 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f34c809e000 tx=0x7f34c8099830 comp rx=0 tx=0).stop 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.443+0000 7f34e576eb80 1 -- 172.21.15.42:0/3332176926 shutdown_connections 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.443+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ba86d09f40 0x55ba86d0c330 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ba86c9f660 0x55ba86ca8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34b6ff5700 0 client.15714 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ba86c99ac0 0x55ba86ca8020 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34e576eb80 1 --2- 172.21.15.42:0/3332176926 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ba86c9ed40 0x55ba86ca8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34e576eb80 1 -- 172.21.15.42:0/3332176926 >> 172.21.15.42:0/3332176926 conn(0x55ba86bc8780 msgr2=0x55ba86c9c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34e576eb80 1 -- 172.21.15.42:0/3332176926 shutdown_connections 2023-12-12T10:00:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34e576eb80 1 -- 172.21.15.42:0/3332176926 wait complete. 2023-12-12T10:00:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34e576eb80 20 client.15714 trim_cache size 0 max 16384 2023-12-12T10:00:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:00:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:00:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:00:00.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:00:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 -- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34bc0a1710 msgr2=0x7f34bc0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34bc0a1710 0x7f34bc0a3bd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34c4005fd0 tx=0x7f34c4005ee0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 -- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d4102700 msgr2=0x7f34d419d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d4102700 0x7f34d419d860 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f34c800b530 tx=0x7f34c800b840 comp rx=0 tx=0).stop 2023-12-12T10:00:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 -- 172.21.15.42:0/4023402134 shutdown_connections 2023-12-12T10:00:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34bc0a1710 0x7f34bc0a3bd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34d410bad0 0x7f34d41979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34d4102700 0x7f34d419d860 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 --2- 172.21.15.42:0/4023402134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34d4103020 0x7f34d419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 -- 172.21.15.42:0/4023402134 >> 172.21.15.42:0/4023402134 conn(0x7f34d40fbea0 msgr2=0x7f34d40fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.444+0000 7f34d9575700 1 -- 172.21.15.42:0/4023402134 shutdown_connections 2023-12-12T10:00:00.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:00.445+0000 7f34d9575700 1 -- 172.21.15.42:0/4023402134 wait complete. 2023-12-12T10:00:00.692 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:00.692 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:00.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:00 smithi173 ceph-mon[103308]: pgmap v44: 129 pgs: 129 active+clean; 1.5 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-12-12T10:00:00.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:00 smithi173 ceph-mon[103308]: overall HEALTH_OK 2023-12-12T10:00:00.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:00 smithi042 ceph-mon[98894]: pgmap v44: 129 pgs: 129 active+clean; 1.5 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-12-12T10:00:00.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:00 smithi042 ceph-mon[98894]: overall HEALTH_OK 2023-12-12T10:00:00.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:00 smithi111 ceph-mon[102565]: pgmap v44: 129 pgs: 129 active+clean; 1.5 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 19 KiB/s wr, 14 op/s 2023-12-12T10:00:00.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:00 smithi111 ceph-mon[102565]: overall HEALTH_OK 2023-12-12T10:00:01.008 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:01.009 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:01.029 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:01.029 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:01.029 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 22 -lt 100 ']' 2023-12-12T10:00:01.029 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 22 + 1 2023-12-12T10:00:01.076 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=23 2023-12-12T10:00:01.076 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:01.115 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:01.115 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:01.116 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:01.117 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:01.117 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:01.442 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:01.442 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:01.580 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:01.580 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:01.582 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:01.583 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:01.583 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 23 -lt 100 ']' 2023-12-12T10:00:01.583 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 23 + 1 2023-12-12T10:00:01.598 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=24 2023-12-12T10:00:01.598 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:01.614 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:01.614 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:01.614 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:01.615 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:01.615 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:01.925 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:01.925 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:02.057 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:02.057 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:02.070 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:02.070 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:02.070 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 24 -lt 100 ']' 2023-12-12T10:00:02.070 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 24 + 1 2023-12-12T10:00:02.097 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=25 2023-12-12T10:00:02.098 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:02.115 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:02.115 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:02.116 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:02.116 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:02.116 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:02.558 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:02.558 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:02.651 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:02.651 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:02.660 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:02.660 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:02.660 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 25 -lt 100 ']' 2023-12-12T10:00:02.660 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 25 + 1 2023-12-12T10:00:02.689 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=26 2023-12-12T10:00:02.689 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:02.715 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:02.715 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:02.716 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:02.716 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:02.716 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:02.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:02 smithi173 ceph-mon[103308]: pgmap v45: 129 pgs: 129 active+clean; 1.5 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-12-12T10:00:02.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:02 smithi042 ceph-mon[98894]: pgmap v45: 129 pgs: 129 active+clean; 1.5 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-12-12T10:00:02.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:02 smithi111 ceph-mon[102565]: pgmap v45: 129 pgs: 129 active+clean; 1.5 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 9 op/s 2023-12-12T10:00:03.075 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:03.075 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:03.215 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:03.215 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:03.228 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:03.228 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:03.228 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 26 -lt 100 ']' 2023-12-12T10:00:03.228 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 26 + 1 2023-12-12T10:00:03.264 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=27 2023-12-12T10:00:03.264 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:03.282 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:03.282 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:03.282 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:03.282 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:03.283 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:03.641 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:03.642 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:03.774 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:03.774 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:03.785 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:03.785 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:03.785 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 27 -lt 100 ']' 2023-12-12T10:00:03.785 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 27 + 1 2023-12-12T10:00:03.806 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=28 2023-12-12T10:00:03.806 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:03.823 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:03.824 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:03.825 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:03.825 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:03.825 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:04.191 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:04.192 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:04.373 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:04.374 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:04.388 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:04.388 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:04.388 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 28 -lt 100 ']' 2023-12-12T10:00:04.388 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 28 + 1 2023-12-12T10:00:04.434 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=29 2023-12-12T10:00:04.434 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:04.457 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:04.457 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:04.457 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:04.457 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:04.458 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:04 smithi173 ceph-mon[103308]: pgmap v46: 129 pgs: 129 active+clean; 1.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 34 op/s 2023-12-12T10:00:04.841 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:04.841 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:04 smithi042 ceph-mon[98894]: pgmap v46: 129 pgs: 129 active+clean; 1.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 34 op/s 2023-12-12T10:00:04.973 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:04.974 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:04.985 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:04.986 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:04.986 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 29 -lt 100 ']' 2023-12-12T10:00:04.986 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 29 + 1 2023-12-12T10:00:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:04 smithi111 ceph-mon[102565]: pgmap v46: 129 pgs: 129 active+clean; 1.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 52 KiB/s wr, 34 op/s 2023-12-12T10:00:05.006 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=30 2023-12-12T10:00:05.006 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:05.024 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:05.024 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:05.025 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:05.025 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:05.025 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:05.375 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:05.375 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:05.490 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:05.490 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:05.503 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:05.503 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:05.503 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 30 -lt 100 ']' 2023-12-12T10:00:05.503 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 30 + 1 2023-12-12T10:00:05.530 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=31 2023-12-12T10:00:05.530 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:05.548 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:05.548 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:05.549 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:05.549 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:05.549 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:05.958 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:05.958 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:06.057 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:06.057 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:06.068 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:06.068 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:06.068 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 31 -lt 100 ']' 2023-12-12T10:00:06.069 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 31 + 1 2023-12-12T10:00:06.088 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=32 2023-12-12T10:00:06.089 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:06.107 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:06.107 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:06.108 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:06.108 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:06.108 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:06.558 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:06.558 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:06.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:06 smithi042 ceph-mon[98894]: pgmap v47: 129 pgs: 129 active+clean; 1.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 KiB/s wr, 29 op/s 2023-12-12T10:00:06.698 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:06.698 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:06.711 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:06.712 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:06.712 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 32 -lt 100 ']' 2023-12-12T10:00:06.712 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 32 + 1 2023-12-12T10:00:06.738 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=33 2023-12-12T10:00:06.738 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:06.757 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:06.757 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:06.758 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:06.758 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:06.758 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:06 smithi173 ceph-mon[103308]: pgmap v47: 129 pgs: 129 active+clean; 1.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 KiB/s wr, 29 op/s 2023-12-12T10:00:06.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:06 smithi111 ceph-mon[102565]: pgmap v47: 129 pgs: 129 active+clean; 1.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 KiB/s wr, 29 op/s 2023-12-12T10:00:07.124 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:07.125 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:07.265 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:07.265 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:07.276 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:07.276 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:07.276 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 33 -lt 100 ']' 2023-12-12T10:00:07.276 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 33 + 1 2023-12-12T10:00:07.297 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=34 2023-12-12T10:00:07.297 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:07.315 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:07.315 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:07.316 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:07.316 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:07.316 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:07.766 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:07.766 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:07.881 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:07.881 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:07.893 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:07.893 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:07.894 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 34 -lt 100 ']' 2023-12-12T10:00:07.894 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 34 + 1 2023-12-12T10:00:07.913 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=35 2023-12-12T10:00:07.913 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:07.939 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:07.940 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:07.940 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:07.940 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:07.940 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:08.307 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:08.308 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:08.448 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:08.448 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:08.459 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:08.459 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:08.459 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 35 -lt 100 ']' 2023-12-12T10:00:08.459 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 35 + 1 2023-12-12T10:00:08.480 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=36 2023-12-12T10:00:08.480 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:08.497 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:08.497 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:08.498 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:08.498 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:08.498 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:08.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:08 smithi042 ceph-mon[98894]: pgmap v48: 129 pgs: 129 active+clean; 1.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 KiB/s wr, 29 op/s 2023-12-12T10:00:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:08 smithi173 ceph-mon[103308]: pgmap v48: 129 pgs: 129 active+clean; 1.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 KiB/s wr, 29 op/s 2023-12-12T10:00:08.899 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:08.899 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:08.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:08 smithi111 ceph-mon[102565]: pgmap v48: 129 pgs: 129 active+clean; 1.6 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 44 KiB/s wr, 29 op/s 2023-12-12T10:00:09.040 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:09.040 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:09.052 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:09.053 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:09.053 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 36 -lt 100 ']' 2023-12-12T10:00:09.053 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 36 + 1 2023-12-12T10:00:09.088 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=37 2023-12-12T10:00:09.088 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:09.114 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:09.114 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:09.115 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:09.115 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:09.115 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:09.524 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:09.524 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:09.673 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:09.673 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:09.684 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:09.684 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:09.684 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 37 -lt 100 ']' 2023-12-12T10:00:09.684 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 37 + 1 2023-12-12T10:00:09.704 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=38 2023-12-12T10:00:09.705 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:09.723 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:09.723 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:09.724 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:09.724 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:09.724 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:10.166 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:10.166 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:10.523 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:10.523 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:10.535 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:10.535 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:10.536 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 38 -lt 100 ']' 2023-12-12T10:00:10.536 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 38 + 1 2023-12-12T10:00:10.563 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=39 2023-12-12T10:00:10.563 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:10.581 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:10.582 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:10.582 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:10.582 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:10.583 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:10.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:10 smithi173 ceph-mon[103308]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 36 op/s 2023-12-12T10:00:10.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:10 smithi173 ceph-mon[103308]: mgrmap e25: x(active, since 92s), standbys: z, y 2023-12-12T10:00:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:10 smithi042 ceph-mon[98894]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 36 op/s 2023-12-12T10:00:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:10 smithi042 ceph-mon[98894]: mgrmap e25: x(active, since 92s), standbys: z, y 2023-12-12T10:00:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:10 smithi111 ceph-mon[102565]: pgmap v49: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 57 KiB/s wr, 36 op/s 2023-12-12T10:00:10.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:10 smithi111 ceph-mon[102565]: mgrmap e25: x(active, since 92s), standbys: z, y 2023-12-12T10:00:11.807 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:11.807 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:11.940 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:11.940 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:11.952 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:11.952 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:11.952 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 39 -lt 100 ']' 2023-12-12T10:00:11.952 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 39 + 1 2023-12-12T10:00:11.996 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=40 2023-12-12T10:00:11.997 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:12.014 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:12.014 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:12.014 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:12.015 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:12.015 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:12.399 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:12.449 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:12.508 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:12.508 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:12.521 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:12.521 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:12.522 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 40 -lt 100 ']' 2023-12-12T10:00:12.522 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 40 + 1 2023-12-12T10:00:12.554 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=41 2023-12-12T10:00:12.555 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:12.596 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:12.597 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:12.598 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:12.598 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:12.598 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:12.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:12 smithi173 ceph-mon[103308]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 48 KiB/s wr, 31 op/s 2023-12-12T10:00:12.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:12 smithi042 ceph-mon[98894]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 48 KiB/s wr, 31 op/s 2023-12-12T10:00:12.990 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:12.991 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:12.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:12 smithi111 ceph-mon[102565]: pgmap v50: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 48 KiB/s wr, 31 op/s 2023-12-12T10:00:13.115 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:13.115 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:13.126 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:13.127 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:13.127 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 41 -lt 100 ']' 2023-12-12T10:00:13.127 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 41 + 1 2023-12-12T10:00:13.155 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=42 2023-12-12T10:00:13.155 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:13.172 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:13.173 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:13.173 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:13.173 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:13.173 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:13.465 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:13.465 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:13.571 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:13.571 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:13.584 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:13.584 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:13.584 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 42 -lt 100 ']' 2023-12-12T10:00:13.584 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 42 + 1 2023-12-12T10:00:13.612 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=43 2023-12-12T10:00:13.612 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:13.653 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:13.653 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:13.654 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:13.654 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:13.654 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:14.182 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:14.182 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:14.272 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:14.272 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:14.287 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:14.287 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:14.287 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 43 -lt 100 ']' 2023-12-12T10:00:14.287 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 43 + 1 2023-12-12T10:00:14.329 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=44 2023-12-12T10:00:14.329 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:14.348 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:14.348 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:14.348 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:14.349 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:14.349 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:14.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:14 smithi173 ceph-mon[103308]: pgmap v51: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 KiB/s wr, 36 op/s 2023-12-12T10:00:14.827 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:14 smithi042 ceph-mon[98894]: pgmap v51: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 KiB/s wr, 36 op/s 2023-12-12T10:00:14.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:14 smithi111 ceph-mon[102565]: pgmap v51: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 56 KiB/s wr, 36 op/s 2023-12-12T10:00:15.015 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:15.015 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:15.330 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:15.330 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:15.343 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:15.343 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:15.343 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 44 -lt 100 ']' 2023-12-12T10:00:15.343 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 44 + 1 2023-12-12T10:00:15.395 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=45 2023-12-12T10:00:15.396 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:15.447 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:15.448 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:15.448 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:15.448 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:15.448 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.270+0000 7f4a89bcd700 1 -- 172.21.15.42:0/3241364668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a841014b0 msgr2=0x7f4a84101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.270+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/3241364668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a841014b0 0x7f4a84101930 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4a780099c0 tx=0x7f4a78009cd0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.270+0000 7f4a89bcd700 1 -- 172.21.15.42:0/3241364668 shutdown_connections 2023-12-12T10:00:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.270+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/3241364668 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a84107570 0x7f4a84107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.270+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/3241364668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a84101e70 0x7f4a8410e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.270+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/3241364668 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a841014b0 0x7f4a84101930 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.270+0000 7f4a89bcd700 1 -- 172.21.15.42:0/3241364668 >> 172.21.15.42:0/3241364668 conn(0x7f4a840fcfa0 msgr2=0x7f4a840ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.270+0000 7f4a89bcd700 1 -- 172.21.15.42:0/3241364668 shutdown_connections 2023-12-12T10:00:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.270+0000 7f4a89bcd700 1 -- 172.21.15.42:0/3241364668 wait complete. 2023-12-12T10:00:16.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.271+0000 7f4a89bcd700 1 Processor -- start 2023-12-12T10:00:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.271+0000 7f4a89bcd700 1 -- start start 2023-12-12T10:00:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a89bcd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a841014b0 0x7f4a8419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a89bcd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a84101e70 0x7f4a8419dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a89bcd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a84107570 0x7f4a841977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a89bcd700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84197e30 con 0x7f4a84101e70 2023-12-12T10:00:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a89bcd700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84197fa0 con 0x7f4a84107570 2023-12-12T10:00:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a89bcd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84198160 con 0x7f4a841014b0 2023-12-12T10:00:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a84101e70 0x7f4a8419dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a84101e70 0x7f4a8419dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36102/0 (socket says 172.21.15.42:36102) 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a82ffd700 1 -- 172.21.15.42:0/4255576173 learned_addr learned my addr 172.21.15.42:0/4255576173 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a837fe700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a841014b0 0x7f4a8419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a83fff700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a84107570 0x7f4a841977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a82ffd700 1 -- 172.21.15.42:0/4255576173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a841014b0 msgr2=0x7f4a8419d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a841014b0 0x7f4a8419d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a82ffd700 1 -- 172.21.15.42:0/4255576173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a84107570 msgr2=0x7f4a841977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a84107570 0x7f4a841977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a82ffd700 1 -- 172.21.15.42:0/4255576173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a780095f0 con 0x7f4a84101e70 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.272+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a84101e70 0x7f4a8419dbd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800b580 tx=0x7f4a78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.273+0000 7f4a80ff9700 1 -- 172.21.15.42:0/4255576173 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7801d070 con 0x7f4a84101e70 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.273+0000 7f4a89bcd700 1 -- 172.21.15.42:0/4255576173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a841983e0 con 0x7f4a84101e70 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.273+0000 7f4a80ff9700 1 -- 172.21.15.42:0/4255576173 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a7800fb30 con 0x7f4a84101e70 2023-12-12T10:00:16.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.273+0000 7f4a80ff9700 1 -- 172.21.15.42:0/4255576173 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78022c80 con 0x7f4a84101e70 2023-12-12T10:00:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.273+0000 7f4a89bcd700 1 -- 172.21.15.42:0/4255576173 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a841a4680 con 0x7f4a84101e70 2023-12-12T10:00:16.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.275+0000 7f4a80ff9700 1 -- 172.21.15.42:0/4255576173 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f4a7800fca0 con 0x7f4a84101e70 2023-12-12T10:00:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.276+0000 7f4a80ff9700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4a6c0a15f0 0x7f4a6c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.276+0000 7f4a80ff9700 1 -- 172.21.15.42:0/4255576173 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4a780ce2b0 con 0x7f4a84101e70 2023-12-12T10:00:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.276+0000 7f4a837fe700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4a6c0a15f0 0x7f4a6c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.276+0000 7f4a837fe700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4a6c0a15f0 0x7f4a6c0a3ab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a74005950 tx=0x7f4a7400b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.374+0000 7f4a95dc6b80 1 Processor -- start 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.374+0000 7f4a95dc6b80 1 -- start start 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.374+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d1bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.374+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d1fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d202d0 0x55f0d1d20730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1c72af0 con 0x55f0d1d1bb80 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1c70410 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1c6bd00 con 0x55f0d1d202d0 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d1bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d1bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36108/0 (socket says 172.21.15.42:36108) 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a837fe700 1 -- 172.21.15.42:0/2107401300 learned_addr learned my addr 172.21.15.42:0/2107401300 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:16.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a83fff700 1 --2- 172.21.15.42:0/2107401300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d202d0 0x55f0d1d20730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/2107401300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d1fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a837fe700 1 -- 172.21.15.42:0/2107401300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d202d0 msgr2=0x55f0d1d20730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a837fe700 1 --2- 172.21.15.42:0/2107401300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d202d0 0x55f0d1d20730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a837fe700 1 -- 172.21.15.42:0/2107401300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d1c4a0 msgr2=0x55f0d1d1fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a837fe700 1 --2- 172.21.15.42:0/2107401300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d1fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a837fe700 1 -- 172.21.15.42:0/2107401300 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d1ae85d0 con 0x55f0d1d1bb80 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.375+0000 7f4a837fe700 1 --2- 172.21.15.42:0/2107401300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d1bf60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a74010e60 tx=0x7f4a74013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc0ab3d5d54c60e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a69ffb700 1 -- 172.21.15.42:0/2107401300 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74011db0 con 0x55f0d1d1bb80 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a69ffb700 1 -- 172.21.15.42:0/2107401300 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a74015070 con 0x55f0d1d1bb80 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a69ffb700 1 -- 172.21.15.42:0/2107401300 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7401c610 con 0x55f0d1d1bb80 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/2107401300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 msgr2=0x55f0d1d1bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/2107401300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d1bf60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4a74010e60 tx=0x7f4a74013000 comp rx=0 tx=0).stop 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/2107401300 shutdown_connections 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/2107401300 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d202d0 0x55f0d1d20730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/2107401300 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d1bf60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/2107401300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d1fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/2107401300 >> 172.21.15.42:0/2107401300 conn(0x55f0d1c46780 msgr2=0x55f0d1d11c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.376+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/2107401300 shutdown_connections 2023-12-12T10:00:16.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.377+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/2107401300 wait complete. 2023-12-12T10:00:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.377+0000 7f4a95dc6b80 1 -- start start 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.377+0000 7f4a95dc6b80 1 Processor -- start 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d27a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d24d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1c25500 con 0x55f0d1d1bb80 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1c251a0 con 0x55f0d1d202d0 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1aea8c0 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a95dc6b80 10 client.0 _ll_register_callbacks cb 0x55f0d1b50a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d27a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d24d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a837fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d27a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36114/0 (socket says 172.21.15.42:36114) 2023-12-12T10:00:16.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a837fe700 1 -- 172.21.15.42:0/1508086215 learned_addr learned my addr 172.21.15.42:0/1508086215 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a83fff700 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a82ffd700 1 -- 172.21.15.42:0/1508086215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 msgr2=0x55f0d1d253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d253d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a82ffd700 1 -- 172.21.15.42:0/1508086215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 msgr2=0x55f0d1d27a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d27a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a82ffd700 1 -- 172.21.15.42:0/1508086215 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a74010b10 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a837fe700 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d27a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d24d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a780cb6b0 tx=0x7f4a7809b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e08799fe4b0584c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.378+0000 7f4a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a62ffd700 1 -- 172.21.15.42:0/1508086215 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7809f120 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1508086215 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0d1d0a730 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a62ffd700 1 -- 172.21.15.42:0/1508086215 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4a780a32d0 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a62ffd700 1 -- 172.21.15.42:0/1508086215 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a780b9170 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a95dc6b80 10 client.25375 fetch_fsmap learned FSMap version 27 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1508086215 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0d1a94d80 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a62ffd700 1 -- 172.21.15.42:0/1508086215 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a78064080 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a62ffd700 1 -- 172.21.15.42:0/1508086215 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a780aed20 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a95dc6b80 10 client.25375 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a95dc6b80 10 client.25375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a95dc6b80 20 client.25375 populate_metadata read hostname 'smithi042' 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f0d1d35020 0x55f0d1d37440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a95dc6b80 4 client.25375 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:00:16.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.379+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1508086215 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0d1ce6ee0 con 0x55f0d1d35020 2023-12-12T10:00:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.380+0000 7f4a837fe700 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f0d1d35020 0x55f0d1d37440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.380+0000 7f4a837fe700 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f0d1d35020 0x55f0d1d37440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9700c58f8b53f06a server_cookie=996b17faa63cc878 in_seq=0 out_seq=0 2023-12-12T10:00:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.380+0000 7f4a62ffd700 10 client.25375 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a62ffd700 1 -- 172.21.15.42:0/1508086215 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f0d1ce6ee0 con 0x55f0d1d35020 2023-12-12T10:00:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a62ffd700 10 client.25375 handle_command_reply: tid=0 2023-12-12T10:00:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 client.25375 shutdown 2023-12-12T10:00:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1508086215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1c4a0 msgr2=0x55f0d1d24d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:16.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d24d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4a780cb6b0 tx=0x7f4a7809b750 comp rx=0 tx=0).stop 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1508086215 shutdown_connections 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f0d1d35020 0x55f0d1d37440 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a62ffd700 0 client.25375 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d27a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1508086215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d24d30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1508086215 >> 172.21.15.42:0/1508086215 conn(0x55f0d1c46780 msgr2=0x55f0d1d16570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1508086215 shutdown_connections 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1508086215 wait complete. 2023-12-12T10:00:16.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.381+0000 7f4a95dc6b80 20 client.25375 trim_cache size 0 max 16384 2023-12-12T10:00:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a95dc6b80 1 Processor -- start 2023-12-12T10:00:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a95dc6b80 1 -- start start 2023-12-12T10:00:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d13910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d27770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d24290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1d28240 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1d0e910 con 0x55f0d1d202d0 2023-12-12T10:00:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1d232a0 con 0x55f0d1d1bb80 2023-12-12T10:00:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d27770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d27770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36124/0 (socket says 172.21.15.42:36124) 2023-12-12T10:00:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.397+0000 7f4a82ffd700 1 -- 172.21.15.42:0/1823924916 learned_addr learned my addr 172.21.15.42:0/1823924916 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a837fe700 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d13910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a83fff700 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d24290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a82ffd700 1 -- 172.21.15.42:0/1823924916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 msgr2=0x55f0d1d13910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:16.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d13910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a82ffd700 1 -- 172.21.15.42:0/1823924916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 msgr2=0x55f0d1d24290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d24290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a82ffd700 1 -- 172.21.15.42:0/1823924916 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0d1a7dc00 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a837fe700 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d13910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a83fff700 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d24290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d27770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4a780cb760 tx=0x7f4a78009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99074d5c78b51ac0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a6a7fc700 1 -- 172.21.15.42:0/1823924916 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a780a32d0 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a6a7fc700 1 -- 172.21.15.42:0/1823924916 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f0d1d0a730 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.398+0000 7f4a6a7fc700 1 -- 172.21.15.42:0/1823924916 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7809a5e0 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1823924916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 msgr2=0x55f0d1d27770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:16.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d27770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4a780cb760 tx=0x7f4a78009cd0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1823924916 shutdown_connections 2023-12-12T10:00:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d24290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d13910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/1823924916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d27770 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1823924916 >> 172.21.15.42:0/1823924916 conn(0x55f0d1c46780 msgr2=0x55f0d1d16e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1823924916 shutdown_connections 2023-12-12T10:00:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/1823924916 wait complete. 2023-12-12T10:00:16.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.399+0000 7f4a95dc6b80 1 -- start start 2023-12-12T10:00:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a95dc6b80 1 Processor -- start 2023-12-12T10:00:16.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d16570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d16ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a95dc6b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d16ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1d8a600 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1d8a770 con 0x55f0d1d202d0 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a95dc6b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0d1d8a930 con 0x55f0d1d1bb80 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a95dc6b80 10 client.0 _ll_register_callbacks cb 0x55f0d1b50a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d16ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d16ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36130/0 (socket says 172.21.15.42:36130) 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a82ffd700 1 -- 172.21.15.42:0/505204999 learned_addr learned my addr 172.21.15.42:0/505204999 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:16.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a83fff700 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d16ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a837fe700 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d16570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a82ffd700 1 -- 172.21.15.42:0/505204999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 msgr2=0x55f0d1d16570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d16570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a82ffd700 1 -- 172.21.15.42:0/505204999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 msgr2=0x55f0d1d16ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d16ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.400+0000 7f4a82ffd700 1 -- 172.21.15.42:0/505204999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0d1a94d80 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a82ffd700 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d16ab0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4a7809af60 tx=0x7f4a7809ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416dce329a42bbcf server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a62ffd700 1 -- 172.21.15.42:0/505204999 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7809b3a0 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a62ffd700 1 -- 172.21.15.42:0/505204999 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4a7809b6d0 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/505204999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a74015070 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a62ffd700 1 -- 172.21.15.42:0/505204999 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a780a32d0 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a95dc6b80 10 client.15732 fetch_fsmap learned FSMap version 27 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/505204999 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f0d1abb1f0 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a62ffd700 1 -- 172.21.15.42:0/505204999 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a74015070 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a62ffd700 1 -- 172.21.15.42:0/505204999 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f4a780aed20 con 0x55f0d1d1c4a0 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a95dc6b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a95dc6b80 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a95dc6b80 20 client.15732 populate_metadata read hostname 'smithi042' 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f0d1d8bb40 0x55f0d1d8df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a95dc6b80 4 client.15732 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.401+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/505204999 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f0d1c31f70 con 0x55f0d1d8bb40 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.402+0000 7f4a837fe700 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f0d1d8bb40 0x55f0d1d8df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.402+0000 7f4a837fe700 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f0d1d8bb40 0x55f0d1d8df30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129480ccdcd676f8 server_cookie=79fc186b435fd6f2 in_seq=0 out_seq=0 2023-12-12T10:00:16.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.402+0000 7f4a62ffd700 10 client.15732 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a62ffd700 1 -- 172.21.15.42:0/505204999 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f0d1c31f70 con 0x55f0d1d8bb40 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a62ffd700 10 client.15732 handle_command_reply: tid=0 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 client.15732 shutdown 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/505204999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 msgr2=0x55f0d1d16ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d16ab0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4a7809af60 tx=0x7f4a7809ada0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/505204999 shutdown_connections 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f0d1d8bb40 0x55f0d1d8df30 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f0d1d202d0 0x55f0d1d16ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a62ffd700 0 client.15732 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f0d1d1bb80 0x55f0d1d16570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 --2- 172.21.15.42:0/505204999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f0d1d1c4a0 0x55f0d1d16ab0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/505204999 >> 172.21.15.42:0/505204999 conn(0x55f0d1c46780 msgr2=0x55f0d1d1e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/505204999 shutdown_connections 2023-12-12T10:00:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 1 -- 172.21.15.42:0/505204999 wait complete. 2023-12-12T10:00:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.430+0000 7f4a95dc6b80 20 client.15732 trim_cache size 0 max 16384 2023-12-12T10:00:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:00:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:00:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:00:16.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:00:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 -- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4a6c0a15f0 msgr2=0x7f4a6c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4a6c0a15f0 0x7f4a6c0a3ab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4a74005950 tx=0x7f4a7400b410 comp rx=0 tx=0).stop 2023-12-12T10:00:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 -- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a84101e70 msgr2=0x7f4a8419dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a84101e70 0x7f4a8419dbd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4a7800b580 tx=0x7f4a78004730 comp rx=0 tx=0).stop 2023-12-12T10:00:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 -- 172.21.15.42:0/4255576173 shutdown_connections 2023-12-12T10:00:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4a6c0a15f0 0x7f4a6c0a3ab0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4a84107570 0x7f4a841977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a841014b0 0x7f4a8419d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 --2- 172.21.15.42:0/4255576173 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4a84101e70 0x7f4a8419dbd0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 -- 172.21.15.42:0/4255576173 >> 172.21.15.42:0/4255576173 conn(0x7f4a840fcfa0 msgr2=0x7f4a840fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 -- 172.21.15.42:0/4255576173 shutdown_connections 2023-12-12T10:00:16.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:16.431+0000 7f4a89bcd700 1 -- 172.21.15.42:0/4255576173 wait complete. 2023-12-12T10:00:16.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:16 smithi173 ceph-mon[103308]: pgmap v52: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-12-12T10:00:16.827 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:16 smithi042 ceph-mon[98894]: pgmap v52: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-12-12T10:00:16.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:16 smithi111 ceph-mon[102565]: pgmap v52: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-12-12T10:00:17.932 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:17.932 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:18.413 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:18.414 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:18.501 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:18.501 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:18.501 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 45 -lt 100 ']' 2023-12-12T10:00:18.502 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 45 + 1 2023-12-12T10:00:18.570 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=46 2023-12-12T10:00:18.571 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:18.622 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:18.622 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:18.623 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:18.623 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:18.623 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:18.902 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:18 smithi042 ceph-mon[98894]: pgmap v53: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-12-12T10:00:18.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:18 smithi111 ceph-mon[102565]: pgmap v53: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-12-12T10:00:19.031 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:19.032 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:19.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:18 smithi173 ceph-mon[103308]: pgmap v53: 129 pgs: 129 active+clean; 1.8 MiB data, 334 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2023-12-12T10:00:19.180 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:19.180 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:19.193 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:19.193 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:19.193 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 46 -lt 100 ']' 2023-12-12T10:00:19.193 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 46 + 1 2023-12-12T10:00:19.230 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=47 2023-12-12T10:00:19.230 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:19.256 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:19.256 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:19.257 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:19.257 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:19.257 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:19.623 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:19.623 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:19.780 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:19.780 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:19.793 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:19.793 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:19.793 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 47 -lt 100 ']' 2023-12-12T10:00:19.793 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 47 + 1 2023-12-12T10:00:19.820 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=48 2023-12-12T10:00:19.820 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:19.846 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:19.847 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:19.848 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:19.848 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:19.848 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:20.664 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:20.665 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:20.863 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:20.863 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:20.878 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:20.878 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:20.878 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 48 -lt 100 ']' 2023-12-12T10:00:20.878 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 48 + 1 2023-12-12T10:00:20.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:20 smithi042 ceph-mon[98894]: pgmap v54: 129 pgs: 129 active+clean; 2.0 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 24 op/s 2023-12-12T10:00:20.911 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=49 2023-12-12T10:00:20.911 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:20.930 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:20.930 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:20.931 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:20.932 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:20.932 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:20.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:20 smithi111 ceph-mon[102565]: pgmap v54: 129 pgs: 129 active+clean; 2.0 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 24 op/s 2023-12-12T10:00:21.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:20 smithi173 ceph-mon[103308]: pgmap v54: 129 pgs: 129 active+clean; 2.0 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 24 op/s 2023-12-12T10:00:21.306 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:21.306 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:21.446 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:21.446 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:21.458 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:21.458 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:21.458 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 49 -lt 100 ']' 2023-12-12T10:00:21.458 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 49 + 1 2023-12-12T10:00:21.486 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=50 2023-12-12T10:00:21.487 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:21.506 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:21.506 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:21.507 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:21.507 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:21.507 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:21.785 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:21.785 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:21.885 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:21.885 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:21.901 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:21.901 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:21.901 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 50 -lt 100 ']' 2023-12-12T10:00:21.901 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 50 + 1 2023-12-12T10:00:21.928 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=51 2023-12-12T10:00:21.928 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:21.946 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:21.947 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:21.947 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:21.947 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:21.947 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:22.272 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:22.273 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:22.413 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:22.413 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:22.425 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:22.425 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:22.425 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 51 -lt 100 ']' 2023-12-12T10:00:22.425 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 51 + 1 2023-12-12T10:00:22.453 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=52 2023-12-12T10:00:22.454 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:22.472 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:22.472 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:22.473 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:22.473 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:22.473 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:22.789 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:22.789 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:22.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:22 smithi042 ceph-mon[98894]: pgmap v55: 129 pgs: 129 active+clean; 2.0 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2023-12-12T10:00:22.946 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:22.947 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:22.958 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:22.958 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:22.958 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 52 -lt 100 ']' 2023-12-12T10:00:22.958 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 52 + 1 2023-12-12T10:00:22.987 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=53 2023-12-12T10:00:22.987 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:22.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:22 smithi111 ceph-mon[102565]: pgmap v55: 129 pgs: 129 active+clean; 2.0 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2023-12-12T10:00:23.005 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:23.005 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:23.006 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:23.006 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:23.006 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:22 smithi173 ceph-mon[103308]: pgmap v55: 129 pgs: 129 active+clean; 2.0 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 17 op/s 2023-12-12T10:00:23.381 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:23.381 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:23.488 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:23.488 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:23.499 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:23.500 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:23.500 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 53 -lt 100 ']' 2023-12-12T10:00:23.500 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 53 + 1 2023-12-12T10:00:23.520 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=54 2023-12-12T10:00:23.520 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:23.538 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:23.538 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:23.541 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:23.541 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:23.542 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:23.906 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:23.906 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:24.039 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:24.039 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:24.052 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:24.052 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:24.052 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 54 -lt 100 ']' 2023-12-12T10:00:24.052 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 54 + 1 2023-12-12T10:00:24.078 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=55 2023-12-12T10:00:24.078 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:24.096 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:24.097 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:24.098 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:24.098 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:24.098 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:24.431 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:24.431 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:24.532 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:24.532 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:24.541 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:24.541 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:24.541 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 55 -lt 100 ']' 2023-12-12T10:00:24.541 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 55 + 1 2023-12-12T10:00:24.570 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=56 2023-12-12T10:00:24.570 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:24.588 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:24.588 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:24.588 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:24.589 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:24.589 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:24.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:24 smithi042 ceph-mon[98894]: pgmap v56: 129 pgs: 129 active+clean; 2.1 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 22 op/s 2023-12-12T10:00:24.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:24 smithi111 ceph-mon[102565]: pgmap v56: 129 pgs: 129 active+clean; 2.1 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 22 op/s 2023-12-12T10:00:25.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:24 smithi173 ceph-mon[103308]: pgmap v56: 129 pgs: 129 active+clean; 2.1 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 22 op/s 2023-12-12T10:00:25.164 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:25.164 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:25.313 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:25.313 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:25.326 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:25.326 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:25.326 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 56 -lt 100 ']' 2023-12-12T10:00:25.326 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 56 + 1 2023-12-12T10:00:25.353 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=57 2023-12-12T10:00:25.353 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:25.370 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:25.371 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:25.371 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:25.372 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:25.372 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:25.747 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:25.747 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:25.912 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:25.913 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:25.932 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:25.933 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:25.933 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 57 -lt 100 ']' 2023-12-12T10:00:25.933 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 57 + 1 2023-12-12T10:00:25.961 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=58 2023-12-12T10:00:25.961 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:25.996 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:25.996 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:25.997 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:25.997 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:25.997 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:26.338 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:26.339 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:26.463 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:26.463 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:26.475 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:26.475 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:26.475 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 58 -lt 100 ']' 2023-12-12T10:00:26.475 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 58 + 1 2023-12-12T10:00:26.502 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=59 2023-12-12T10:00:26.503 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:26.520 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:26.521 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:26.522 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:26.522 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:26.522 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:26.830 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:26.830 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:26.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:26 smithi042 ceph-mon[98894]: pgmap v57: 129 pgs: 129 active+clean; 2.1 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-12T10:00:26.955 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:26.955 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:26.966 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:26.966 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:26.966 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 59 -lt 100 ']' 2023-12-12T10:00:26.966 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 59 + 1 2023-12-12T10:00:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:26 smithi111 ceph-mon[102565]: pgmap v57: 129 pgs: 129 active+clean; 2.1 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-12T10:00:27.027 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=60 2023-12-12T10:00:27.028 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:27.062 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:27.063 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:27.063 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:27.064 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:27.064 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:27.072 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:26 smithi173 ceph-mon[103308]: pgmap v57: 129 pgs: 129 active+clean; 2.1 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-12T10:00:27.381 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:27.381 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:27.487 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:27.487 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:27.500 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:27.500 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:27.500 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 60 -lt 100 ']' 2023-12-12T10:00:27.500 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 60 + 1 2023-12-12T10:00:27.527 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=61 2023-12-12T10:00:27.528 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:27.547 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:27.547 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:27.547 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:27.548 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:27.548 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:27.889 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:27.889 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:28.021 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:28.021 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:28.032 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:28.032 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:28.032 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 61 -lt 100 ']' 2023-12-12T10:00:28.032 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 61 + 1 2023-12-12T10:00:28.061 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=62 2023-12-12T10:00:28.061 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:28.079 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:28.079 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:28.080 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:28.081 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:28.081 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:28.513 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:28.514 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:28.596 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:28.596 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:28.608 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:28.609 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:28.609 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 62 -lt 100 ']' 2023-12-12T10:00:28.609 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 62 + 1 2023-12-12T10:00:28.639 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=63 2023-12-12T10:00:28.639 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:28.674 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:28.675 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:28.677 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:28.677 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:28.677 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:28.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:28 smithi042 ceph-mon[98894]: pgmap v58: 129 pgs: 129 active+clean; 2.1 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-12T10:00:28.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:28 smithi111 ceph-mon[102565]: pgmap v58: 129 pgs: 129 active+clean; 2.1 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-12T10:00:29.013 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:29.013 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:28 smithi173 ceph-mon[103308]: pgmap v58: 129 pgs: 129 active+clean; 2.1 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 22 KiB/s wr, 17 op/s 2023-12-12T10:00:29.204 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:29.204 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:29.224 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:29.224 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:29.224 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 63 -lt 100 ']' 2023-12-12T10:00:29.224 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 63 + 1 2023-12-12T10:00:29.253 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=64 2023-12-12T10:00:29.253 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:29.288 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:29.288 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:29.289 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:29.290 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:29.290 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:29.663 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:29.663 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:29.820 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:29.821 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:29.841 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:29.842 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:29.842 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 64 -lt 100 ']' 2023-12-12T10:00:29.842 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 64 + 1 2023-12-12T10:00:29.877 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=65 2023-12-12T10:00:29.877 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:29.906 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:29.906 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:29.907 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:29.907 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:29.907 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:30.646 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:30.647 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:30.763 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:30.763 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:30.800 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:30.800 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:30.800 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 65 -lt 100 ']' 2023-12-12T10:00:30.800 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 65 + 1 2023-12-12T10:00:30.885 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=66 2023-12-12T10:00:30.885 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:30.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:30 smithi042 ceph-mon[98894]: pgmap v59: 129 pgs: 129 active+clean; 2.2 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 22 op/s 2023-12-12T10:00:30.922 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:30.923 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:30.923 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:30.923 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:30.924 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:30.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:30 smithi111 ceph-mon[102565]: pgmap v59: 129 pgs: 129 active+clean; 2.2 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 22 op/s 2023-12-12T10:00:31.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:30 smithi173 ceph-mon[103308]: pgmap v59: 129 pgs: 129 active+clean; 2.2 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 22 op/s 2023-12-12T10:00:31.521 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:31.521 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:31.762 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:31.762 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:31.773 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:31.774 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:31.774 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 66 -lt 100 ']' 2023-12-12T10:00:31.774 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 66 + 1 2023-12-12T10:00:31.818 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=67 2023-12-12T10:00:31.819 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:31.889 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:31.889 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:31.890 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:31.891 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:31.891 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.151+0000 7fb608875700 1 -- 172.21.15.42:0/784095273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6041075a0 msgr2=0x7fb604107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.151+0000 7fb608875700 1 --2- 172.21.15.42:0/784095273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6041075a0 0x7fb604107980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009920 tx=0x7fb5fc009c30 comp rx=0 tx=0).stop 2023-12-12T10:00:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.151+0000 7fb608875700 1 -- 172.21.15.42:0/784095273 shutdown_connections 2023-12-12T10:00:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.151+0000 7fb608875700 1 --2- 172.21.15.42:0/784095273 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb6041075a0 0x7fb604107980 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.151+0000 7fb608875700 1 --2- 172.21.15.42:0/784095273 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb604101ea0 0x7fb60410e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.151+0000 7fb608875700 1 --2- 172.21.15.42:0/784095273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6041014e0 0x7fb604101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.151+0000 7fb608875700 1 -- 172.21.15.42:0/784095273 >> 172.21.15.42:0/784095273 conn(0x7fb6040fcfd0 msgr2=0x7fb6040ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.151+0000 7fb608875700 1 -- 172.21.15.42:0/784095273 shutdown_connections 2023-12-12T10:00:32.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.152+0000 7fb608875700 1 -- 172.21.15.42:0/784095273 wait complete. 2023-12-12T10:00:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.153+0000 7fb608875700 1 Processor -- start 2023-12-12T10:00:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.153+0000 7fb608875700 1 -- start start 2023-12-12T10:00:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.153+0000 7fb608875700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6041014e0 0x7fb60419d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.153+0000 7fb608875700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb604101ea0 0x7fb60419db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.153+0000 7fb608875700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6041075a0 0x7fb604197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb608875700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb604197dc0 con 0x7fb6041014e0 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb608875700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb604197f30 con 0x7fb604101ea0 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb608875700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6041980f0 con 0x7fb6041075a0 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb60259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6041014e0 0x7fb60419d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb60259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6041014e0 0x7fb60419d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36142/0 (socket says 172.21.15.42:36142) 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb60259c700 1 -- 172.21.15.42:0/1577927991 learned_addr learned my addr 172.21.15.42:0/1577927991 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb601d9b700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb604101ea0 0x7fb60419db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb602d9d700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6041075a0 0x7fb604197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb60259c700 1 -- 172.21.15.42:0/1577927991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6041075a0 msgr2=0x7fb604197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb60259c700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6041075a0 0x7fb604197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb60259c700 1 -- 172.21.15.42:0/1577927991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb604101ea0 msgr2=0x7fb60419db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:32.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb60259c700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb604101ea0 0x7fb60419db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb60259c700 1 -- 172.21.15.42:0/1577927991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0095f0 con 0x7fb6041014e0 2023-12-12T10:00:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb601d9b700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb604101ea0 0x7fb60419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:00:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb602d9d700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6041075a0 0x7fb604197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:00:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.154+0000 7fb60259c700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6041014e0 0x7fb60419d630 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009230 tx=0x7fb5fc00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.155+0000 7fb5fb7fe700 1 -- 172.21.15.42:0/1577927991 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc01c070 con 0x7fb6041014e0 2023-12-12T10:00:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.155+0000 7fb5fb7fe700 1 -- 172.21.15.42:0/1577927991 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5fc00bb20 con 0x7fb6041014e0 2023-12-12T10:00:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.155+0000 7fb608875700 1 -- 172.21.15.42:0/1577927991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5fc009520 con 0x7fb6041014e0 2023-12-12T10:00:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.155+0000 7fb5fb7fe700 1 -- 172.21.15.42:0/1577927991 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc004930 con 0x7fb6041014e0 2023-12-12T10:00:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.155+0000 7fb608875700 1 -- 172.21.15.42:0/1577927991 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6041a4440 con 0x7fb6041014e0 2023-12-12T10:00:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.157+0000 7fb5fb7fe700 1 -- 172.21.15.42:0/1577927991 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fb5fc020070 con 0x7fb6041014e0 2023-12-12T10:00:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.158+0000 7fb5fb7fe700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fb5f00a15f0 0x7fb5f00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.158+0000 7fb5fb7fe700 1 -- 172.21.15.42:0/1577927991 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb5fc0ce150 con 0x7fb6041014e0 2023-12-12T10:00:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.158+0000 7fb601d9b700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fb5f00a15f0 0x7fb5f00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.159+0000 7fb601d9b700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fb5f00a15f0 0x7fb5f00a3ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb604198e70 tx=0x7fb5ec009310 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.233+0000 7fb614a6eb80 1 Processor -- start 2023-12-12T10:00:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.233+0000 7fb614a6eb80 1 -- start start 2023-12-12T10:00:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc99b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc995e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 0x5625dc9a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc8ecaf0 con 0x5625dc995990 2023-12-12T10:00:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc8ea410 con 0x5625dc996350 2023-12-12T10:00:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc8e5d00 con 0x5625dc99b1a0 2023-12-12T10:00:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb601d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc995e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb601d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc995e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36152/0 (socket says 172.21.15.42:36152) 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb601d9b700 1 -- 172.21.15.42:0/3204127633 learned_addr learned my addr 172.21.15.42:0/3204127633 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb60259c700 1 --2- 172.21.15.42:0/3204127633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc99b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb602d9d700 1 --2- 172.21.15.42:0/3204127633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 0x5625dc9a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb601d9b700 1 -- 172.21.15.42:0/3204127633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 msgr2=0x5625dc99b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb601d9b700 1 --2- 172.21.15.42:0/3204127633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc99b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb601d9b700 1 -- 172.21.15.42:0/3204127633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 msgr2=0x5625dc9a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb601d9b700 1 --2- 172.21.15.42:0/3204127633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 0x5625dc9a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.234+0000 7fb601d9b700 1 -- 172.21.15.42:0/3204127633 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625dc7625d0 con 0x5625dc995990 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.235+0000 7fb601d9b700 1 --2- 172.21.15.42:0/3204127633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc995e10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec004390 tx=0x7fb5ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8e95fc37fb0337 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.235+0000 7fb5f8ff9700 1 -- 172.21.15.42:0/3204127633 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec003d10 con 0x5625dc995990 2023-12-12T10:00:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.235+0000 7fb5f8ff9700 1 -- 172.21.15.42:0/3204127633 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5ec003e70 con 0x5625dc995990 2023-12-12T10:00:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.235+0000 7fb5f8ff9700 1 -- 172.21.15.42:0/3204127633 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0267a0 con 0x5625dc995990 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 -- 172.21.15.42:0/3204127633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 msgr2=0x5625dc995e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/3204127633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc995e10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec004390 tx=0x7fb5ec014000 comp rx=0 tx=0).stop 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 -- 172.21.15.42:0/3204127633 shutdown_connections 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/3204127633 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 0x5625dc9a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/3204127633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc99b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/3204127633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc995e10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 -- 172.21.15.42:0/3204127633 >> 172.21.15.42:0/3204127633 conn(0x5625dc8c0780 msgr2=0x5625dc98bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 -- 172.21.15.42:0/3204127633 shutdown_connections 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 -- 172.21.15.42:0/3204127633 wait complete. 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.236+0000 7fb614a6eb80 1 -- start start 2023-12-12T10:00:32.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb614a6eb80 1 Processor -- start 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc9a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 0x5625dc99f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc99f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc89f500 con 0x5625dc995990 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc89f1a0 con 0x5625dc996350 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc7648c0 con 0x5625dc99b1a0 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb614a6eb80 10 client.0 _ll_register_callbacks cb 0x5625dc7caa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb60259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc9a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb60259c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc9a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36156/0 (socket says 172.21.15.42:36156) 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb60259c700 1 -- 172.21.15.42:0/2916055698 learned_addr learned my addr 172.21.15.42:0/2916055698 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.237+0000 7fb601d9b700 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 0x5625dc99f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb602d9d700 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc99f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb60259c700 1 -- 172.21.15.42:0/2916055698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 msgr2=0x5625dc99f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb60259c700 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc99f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb60259c700 1 -- 172.21.15.42:0/2916055698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 msgr2=0x5625dc99f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb60259c700 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 0x5625dc99f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb60259c700 1 -- 172.21.15.42:0/2916055698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec017070 con 0x5625dc995990 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb60259c700 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc9a1e90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc097310 tx=0x7fb5fc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ac1ef9039d2477 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb5e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2916055698 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc09b270 con 0x5625dc995990 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2916055698 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb5fc005250 con 0x5625dc995990 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2916055698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625dc984730 con 0x5625dc995990 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2916055698 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5fc09ae10 con 0x5625dc995990 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb614a6eb80 10 client.15744 fetch_fsmap learned FSMap version 27 2023-12-12T10:00:32.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.238+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2916055698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5625dc70ed80 con 0x5625dc995990 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.239+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2916055698 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5fc0b7080 con 0x5625dc995990 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.239+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2916055698 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb5fc0a2c80 con 0x5625dc995990 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.239+0000 7fb614a6eb80 10 client.15744 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.239+0000 7fb614a6eb80 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.239+0000 7fb614a6eb80 20 client.15744 populate_metadata read hostname 'smithi042' 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.239+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625dc9af1d0 0x5625dc9b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.239+0000 7fb614a6eb80 4 client.15744 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.239+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2916055698 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625dc960ee0 con 0x5625dc9af1d0 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.239+0000 7fb601d9b700 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625dc9af1d0 0x5625dc9b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.240+0000 7fb601d9b700 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625dc9af1d0 0x5625dc9b15c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded0d81046f25b50 server_cookie=59e8bab314d0f390 in_seq=0 out_seq=0 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.240+0000 7fb5e9ffb700 10 client.15744 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:32.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2916055698 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625dc960ee0 con 0x5625dc9af1d0 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb5e9ffb700 10 client.15744 handle_command_reply: tid=0 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 client.15744 shutdown 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2916055698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 msgr2=0x5625dc9a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc9a1e90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc097310 tx=0x7fb5fc009c30 comp rx=0 tx=0).stop 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2916055698 shutdown_connections 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc99f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc995990 0x5625dc9a1e90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625dc9af1d0 0x5625dc9b15c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2916055698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc996350 0x5625dc99f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb5e9ffb700 0 client.15744 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2916055698 >> 172.21.15.42:0/2916055698 conn(0x5625dc8c0780 msgr2=0x5625dc990570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2916055698 shutdown_connections 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2916055698 wait complete. 2023-12-12T10:00:32.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.241+0000 7fb614a6eb80 20 client.15744 trim_cache size 0 max 16384 2023-12-12T10:00:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.257+0000 7fb614a6eb80 1 Processor -- start 2023-12-12T10:00:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.257+0000 7fb614a6eb80 1 -- start start 2023-12-12T10:00:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc9b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc996350 0x5625dc99cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc99b1a0 0x5625dc9b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc9a0170 con 0x5625dc99b1a0 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc99a020 con 0x5625dc995990 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc99b580 con 0x5625dc996350 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb602d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc99b1a0 0x5625dc9b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb60259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc9b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb60259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc9b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39224/0 (socket says 172.21.15.42:39224) 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb602d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc99b1a0 0x5625dc9b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36168/0 (socket says 172.21.15.42:36168) 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb60259c700 1 -- 172.21.15.42:0/437079974 learned_addr learned my addr 172.21.15.42:0/437079974 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb601d9b700 1 --2- 172.21.15.42:0/437079974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc996350 0x5625dc99cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb602d9d700 1 -- 172.21.15.42:0/437079974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc996350 msgr2=0x5625dc99cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:32.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb602d9d700 1 --2- 172.21.15.42:0/437079974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc996350 0x5625dc99cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb602d9d700 1 -- 172.21.15.42:0/437079974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 msgr2=0x5625dc9b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb602d9d700 1 --2- 172.21.15.42:0/437079974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc9b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb602d9d700 1 -- 172.21.15.42:0/437079974 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625dc6f7c00 con 0x5625dc99b1a0 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.258+0000 7fb60259c700 1 --2- 172.21.15.42:0/437079974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc9b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.259+0000 7fb602d9d700 1 --2- 172.21.15.42:0/437079974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc99b1a0 0x5625dc9b5ec0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40094c0 tx=0x7fb5f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dbf32f0908b50e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.259+0000 7fb5f97fa700 1 -- 172.21.15.42:0/437079974 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f400e030 con 0x5625dc99b1a0 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.259+0000 7fb5f97fa700 1 -- 172.21.15.42:0/437079974 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5f400f040 con 0x5625dc99b1a0 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.259+0000 7fb5f97fa700 1 -- 172.21.15.42:0/437079974 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4005280 con 0x5625dc99b1a0 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.259+0000 7fb614a6eb80 1 -- 172.21.15.42:0/437079974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc99b1a0 msgr2=0x5625dc9b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.259+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/437079974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc99b1a0 0x5625dc9b5ec0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb5f40094c0 tx=0x7fb5f400b040 comp rx=0 tx=0).stop 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 -- 172.21.15.42:0/437079974 shutdown_connections 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/437079974 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc9b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/437079974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc99b1a0 0x5625dc9b5ec0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/437079974 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc996350 0x5625dc99cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 -- 172.21.15.42:0/437079974 >> 172.21.15.42:0/437079974 conn(0x5625dc8c0780 msgr2=0x5625dc9a1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 -- 172.21.15.42:0/437079974 shutdown_connections 2023-12-12T10:00:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 -- 172.21.15.42:0/437079974 wait complete. 2023-12-12T10:00:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 -- start start 2023-12-12T10:00:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 Processor -- start 2023-12-12T10:00:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.260+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc990800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc996350 0x5625dc990d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb614a6eb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc9a5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc9a5ae0 con 0x5625dc996350 2023-12-12T10:00:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc9a5c20 con 0x5625dc995990 2023-12-12T10:00:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb614a6eb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625dc9a5de0 con 0x5625dc99b1a0 2023-12-12T10:00:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb614a6eb80 10 client.0 _ll_register_callbacks cb 0x5625dc7caa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:00:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb601d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc996350 0x5625dc990d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb601d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc996350 0x5625dc990d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36172/0 (socket says 172.21.15.42:36172) 2023-12-12T10:00:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb601d9b700 1 -- 172.21.15.42:0/2245483676 learned_addr learned my addr 172.21.15.42:0/2245483676 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb60259c700 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc990800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb601d9b700 1 -- 172.21.15.42:0/2245483676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 msgr2=0x5625dc9a5560 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-12T10:00:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb601d9b700 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc9a5560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb601d9b700 1 -- 172.21.15.42:0/2245483676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 msgr2=0x5625dc990800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb601d9b700 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc990800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb601d9b700 1 -- 172.21.15.42:0/2245483676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625dc70ed80 con 0x5625dc996350 2023-12-12T10:00:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb601d9b700 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc996350 0x5625dc990d40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec004390 tx=0x7fb5ec003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11195901e9e013c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.261+0000 7fb5e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:00:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2245483676 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec02c040 con 0x5625dc996350 2023-12-12T10:00:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2245483676 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb5ec031780 con 0x5625dc996350 2023-12-12T10:00:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2245483676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5ec003e70 con 0x5625dc996350 2023-12-12T10:00:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2245483676 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5ec01e420 con 0x5625dc996350 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb614a6eb80 10 client.15750 fetch_fsmap learned FSMap version 27 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2245483676 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb5f40091a0 con 0x5625dc996350 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2245483676 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb5ec003e70 con 0x5625dc996350 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2245483676 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7fb5ec024020 con 0x5625dc996350 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb614a6eb80 10 client.15750 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb614a6eb80 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb614a6eb80 20 client.15750 populate_metadata read hostname 'smithi042' 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625dca06530 0x5625dca08950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb614a6eb80 4 client.15750 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.262+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2245483676 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5625dc8abf70 con 0x5625dca06530 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.263+0000 7fb60259c700 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625dca06530 0x5625dca08950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.263+0000 7fb60259c700 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625dca06530 0x5625dca08950 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f08f23a8c8ce67 server_cookie=6f0a6f95f01fbd81 in_seq=0 out_seq=0 2023-12-12T10:00:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.263+0000 7fb5e9ffb700 10 client.15750 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.296+0000 7fb5e9ffb700 1 -- 172.21.15.42:0/2245483676 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5625dc8abf70 con 0x5625dca06530 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb5e9ffb700 10 client.15750 handle_command_reply: tid=0 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 client.15750 shutdown 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2245483676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc996350 msgr2=0x5625dc990d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc996350 0x5625dc990d40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec004390 tx=0x7fb5ec003b20 comp rx=0 tx=0).stop 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2245483676 shutdown_connections 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625dca06530 0x5625dca08950 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625dc995990 0x5625dc990800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625dc99b1a0 0x5625dc9a5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 --2- 172.21.15.42:0/2245483676 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625dc996350 0x5625dc990d40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb5e9ffb700 0 client.15750 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2245483676 >> 172.21.15.42:0/2245483676 conn(0x5625dc8c0780 msgr2=0x5625dc9994f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2245483676 shutdown_connections 2023-12-12T10:00:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 1 -- 172.21.15.42:0/2245483676 wait complete. 2023-12-12T10:00:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.297+0000 7fb614a6eb80 20 client.15750 trim_cache size 0 max 16384 2023-12-12T10:00:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:00:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:00:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:00:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:00:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 -- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fb5f00a15f0 msgr2=0x7fb5f00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fb5f00a15f0 0x7fb5f00a3ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb604198e70 tx=0x7fb5ec009310 comp rx=0 tx=0).stop 2023-12-12T10:00:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 -- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6041014e0 msgr2=0x7fb60419d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6041014e0 0x7fb60419d630 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc009230 tx=0x7fb5fc00fa00 comp rx=0 tx=0).stop 2023-12-12T10:00:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 -- 172.21.15.42:0/1577927991 shutdown_connections 2023-12-12T10:00:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6041075a0 0x7fb604197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb6041014e0 0x7fb60419d630 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fb5f00a15f0 0x7fb5f00a3ab0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 --2- 172.21.15.42:0/1577927991 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb604101ea0 0x7fb60419db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 -- 172.21.15.42:0/1577927991 >> 172.21.15.42:0/1577927991 conn(0x7fb6040fcfd0 msgr2=0x7fb6040fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 -- 172.21.15.42:0/1577927991 shutdown_connections 2023-12-12T10:00:32.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:32.298+0000 7fb608875700 1 -- 172.21.15.42:0/1577927991 wait complete. 2023-12-12T10:00:32.455 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:32.455 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:32.563 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:32.563 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:32.574 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:32.574 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:32.574 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 67 -lt 100 ']' 2023-12-12T10:00:32.574 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 67 + 1 2023-12-12T10:00:32.602 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=68 2023-12-12T10:00:32.602 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:32.628 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:32.629 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:32.629 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:32.630 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:32.630 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:32.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:32 smithi042 ceph-mon[98894]: pgmap v60: 129 pgs: 129 active+clean; 2.2 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-12-12T10:00:32.880 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:32.919 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:32.967 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:32.967 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:32.973 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:32.973 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:32.973 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 68 -lt 100 ']' 2023-12-12T10:00:32.973 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 68 + 1 2023-12-12T10:00:32.980 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=69 2023-12-12T10:00:32.981 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:32.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:32 smithi111 ceph-mon[102565]: pgmap v60: 129 pgs: 129 active+clean; 2.2 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-12-12T10:00:33.003 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:33.003 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:33.004 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:33.004 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:33.005 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:32 smithi173 ceph-mon[103308]: pgmap v60: 129 pgs: 129 active+clean; 2.2 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 20 KiB/s wr, 9 op/s 2023-12-12T10:00:33.177 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:33.177 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:33.253 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:33.253 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:33.264 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:33.264 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:33.264 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 69 -lt 100 ']' 2023-12-12T10:00:33.264 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 69 + 1 2023-12-12T10:00:33.285 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=70 2023-12-12T10:00:33.285 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:33.303 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:33.303 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:33.304 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:33.304 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:33.304 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:33.696 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:33.696 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:33.795 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:33.795 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:33.807 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:33.807 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:33.808 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 70 -lt 100 ']' 2023-12-12T10:00:33.808 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 70 + 1 2023-12-12T10:00:33.852 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=71 2023-12-12T10:00:33.852 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:33.882 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:33.882 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:33.883 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:33.883 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:33.883 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:34.213 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:34.213 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:34.320 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:34.320 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:34.331 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:34.331 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:34.331 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 71 -lt 100 ']' 2023-12-12T10:00:34.331 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 71 + 1 2023-12-12T10:00:34.352 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=72 2023-12-12T10:00:34.352 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:34.370 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:34.370 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:34.371 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:34.371 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:34.371 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:34.712 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:34.713 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:34.836 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:34.837 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:34.849 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:34.849 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:34.849 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 72 -lt 100 ']' 2023-12-12T10:00:34.849 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 72 + 1 2023-12-12T10:00:34.876 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=73 2023-12-12T10:00:34.877 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:34.895 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:34.895 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:34.896 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:34.896 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:34.896 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:34.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:34 smithi042 ceph-mon[98894]: pgmap v61: 129 pgs: 129 active+clean; 2.3 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 20 op/s 2023-12-12T10:00:34.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:34 smithi111 ceph-mon[102565]: pgmap v61: 129 pgs: 129 active+clean; 2.3 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 20 op/s 2023-12-12T10:00:35.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:34 smithi173 ceph-mon[103308]: pgmap v61: 129 pgs: 129 active+clean; 2.3 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 20 op/s 2023-12-12T10:00:35.504 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:35.504 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:35.653 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:35.653 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:35.665 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:35.665 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:35.665 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 73 -lt 100 ']' 2023-12-12T10:00:35.665 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 73 + 1 2023-12-12T10:00:35.693 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=74 2023-12-12T10:00:35.693 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:35.711 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:35.711 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:35.712 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:35.712 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:35.712 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:36.054 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:36.054 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:36.153 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:36.153 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:36.165 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:36.166 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:36.166 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 74 -lt 100 ']' 2023-12-12T10:00:36.166 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 74 + 1 2023-12-12T10:00:36.193 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=75 2023-12-12T10:00:36.193 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:36.211 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:36.211 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:36.212 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:36.212 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:36.212 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:36.471 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:36.471 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:36.578 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:36.578 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:36.589 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:36.590 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:36.590 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 75 -lt 100 ']' 2023-12-12T10:00:36.590 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 75 + 1 2023-12-12T10:00:36.609 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=76 2023-12-12T10:00:36.610 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:36.628 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:36.628 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:36.629 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:36.629 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:36.630 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:36.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:36 smithi042 ceph-mon[98894]: pgmap v62: 129 pgs: 129 active+clean; 2.3 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-12-12T10:00:36.979 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:36.979 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:36.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:36 smithi111 ceph-mon[102565]: pgmap v62: 129 pgs: 129 active+clean; 2.3 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-12-12T10:00:37.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:36 smithi173 ceph-mon[103308]: pgmap v62: 129 pgs: 129 active+clean; 2.3 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-12-12T10:00:37.086 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:37.086 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:37.111 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:37.111 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:37.111 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 76 -lt 100 ']' 2023-12-12T10:00:37.111 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 76 + 1 2023-12-12T10:00:37.138 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=77 2023-12-12T10:00:37.138 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:37.161 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:37.162 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:37.162 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:37.162 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:37.163 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:37.537 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:37.537 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:37.686 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:37.686 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:37.718 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:37.718 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:37.718 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 77 -lt 100 ']' 2023-12-12T10:00:37.719 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 77 + 1 2023-12-12T10:00:37.784 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=78 2023-12-12T10:00:37.784 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:37.811 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:37.811 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:37.812 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:37.812 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:37.812 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:38.179 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:38.179 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:38.277 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:38.278 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:38.298 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:38.298 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:38.298 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 78 -lt 100 ']' 2023-12-12T10:00:38.298 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 78 + 1 2023-12-12T10:00:38.335 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=79 2023-12-12T10:00:38.335 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:38.353 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:38.353 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:38.353 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:38.354 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:38.354 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:38.795 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:38.796 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:38.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:38 smithi042 ceph-mon[98894]: pgmap v63: 129 pgs: 129 active+clean; 2.3 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-12-12T10:00:38.927 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:38.928 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:38.950 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:38.950 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:38.951 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 79 -lt 100 ']' 2023-12-12T10:00:38.951 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 79 + 1 2023-12-12T10:00:38.988 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=80 2023-12-12T10:00:38.988 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:38.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:38 smithi111 ceph-mon[102565]: pgmap v63: 129 pgs: 129 active+clean; 2.3 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-12-12T10:00:39.011 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:39.011 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:39.012 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:39.012 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:39.012 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:39.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:38 smithi173 ceph-mon[103308]: pgmap v63: 129 pgs: 129 active+clean; 2.3 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2023-12-12T10:00:39.387 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:39.387 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:39.502 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:39.503 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:39.514 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:39.514 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:39.514 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 80 -lt 100 ']' 2023-12-12T10:00:39.514 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 80 + 1 2023-12-12T10:00:39.551 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=81 2023-12-12T10:00:39.551 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:39.577 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:39.577 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:39.579 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:39.579 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:39.579 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:40.020 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:40.020 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:40.361 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:40.361 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:40.373 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:40.373 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:40.373 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 81 -lt 100 ']' 2023-12-12T10:00:40.373 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 81 + 1 2023-12-12T10:00:40.401 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=82 2023-12-12T10:00:40.401 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:40.427 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:40.427 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:40.428 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:40.428 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:40.428 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:40.745 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:40.745 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:40.844 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:40.844 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:40.856 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:40.856 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:40.856 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 82 -lt 100 ']' 2023-12-12T10:00:40.856 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 82 + 1 2023-12-12T10:00:40.884 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=83 2023-12-12T10:00:40.884 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:40 smithi042 ceph-mon[98894]: pgmap v64: 129 pgs: 129 active+clean; 2.4 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 21 op/s 2023-12-12T10:00:40.911 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:40.911 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:40.912 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:40.912 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:40.912 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:40.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:40 smithi111 ceph-mon[102565]: pgmap v64: 129 pgs: 129 active+clean; 2.4 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 21 op/s 2023-12-12T10:00:41.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:40 smithi173 ceph-mon[103308]: pgmap v64: 129 pgs: 129 active+clean; 2.4 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 21 op/s 2023-12-12T10:00:41.215 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:41.215 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:41.311 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:41.311 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:41.322 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:41.322 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:41.322 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 83 -lt 100 ']' 2023-12-12T10:00:41.322 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 83 + 1 2023-12-12T10:00:41.343 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=84 2023-12-12T10:00:41.343 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:41.360 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:41.361 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:41.361 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:41.362 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:41.362 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:41.679 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:41.679 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:41.794 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:41.794 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:41.807 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:41.807 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:41.807 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 84 -lt 100 ']' 2023-12-12T10:00:41.807 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 84 + 1 2023-12-12T10:00:41.834 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=85 2023-12-12T10:00:41.834 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:41.852 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:41.852 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:41.852 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:41.853 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:41.853 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:42.262 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:42.262 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:42.377 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:42.377 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:42.389 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:42.389 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:42.389 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 85 -lt 100 ']' 2023-12-12T10:00:42.389 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 85 + 1 2023-12-12T10:00:42.409 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=86 2023-12-12T10:00:42.409 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:42.427 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:42.427 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:42.428 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:42.428 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:42.428 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:42.778 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:42.779 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:42.885 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:42.886 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:42.897 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:42.897 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:42.897 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 86 -lt 100 ']' 2023-12-12T10:00:42.897 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 86 + 1 2023-12-12T10:00:42.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:42 smithi042 ceph-mon[98894]: pgmap v65: 129 pgs: 129 active+clean; 2.4 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 16 op/s 2023-12-12T10:00:42.917 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=87 2023-12-12T10:00:42.917 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:42.944 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:42.944 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:42.945 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:42.945 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:42.946 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:42 smithi111 ceph-mon[102565]: pgmap v65: 129 pgs: 129 active+clean; 2.4 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 16 op/s 2023-12-12T10:00:43.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:42 smithi173 ceph-mon[103308]: pgmap v65: 129 pgs: 129 active+clean; 2.4 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 25 KiB/s wr, 16 op/s 2023-12-12T10:00:43.328 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:43.328 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:43.444 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:43.444 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:43.466 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:43.466 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:43.466 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 87 -lt 100 ']' 2023-12-12T10:00:43.466 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 87 + 1 2023-12-12T10:00:43.492 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=88 2023-12-12T10:00:43.493 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:43.511 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:43.511 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:43.512 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:43.512 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:43.512 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:43.994 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:43.995 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:44.077 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:44.078 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:44.103 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:44.103 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:44.103 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 88 -lt 100 ']' 2023-12-12T10:00:44.103 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 88 + 1 2023-12-12T10:00:44.159 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=89 2023-12-12T10:00:44.159 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:44.165 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:44.165 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:44.166 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:44.166 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:44.167 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:44.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:44 smithi042 ceph-mon[98894]: pgmap v66: 129 pgs: 129 active+clean; 2.6 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 21 op/s 2023-12-12T10:00:44.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:44 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:00:44.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:44 smithi111 ceph-mon[102565]: pgmap v66: 129 pgs: 129 active+clean; 2.6 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 21 op/s 2023-12-12T10:00:44.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:44 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:00:45.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:44 smithi173 ceph-mon[103308]: pgmap v66: 129 pgs: 129 active+clean; 2.6 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 37 KiB/s wr, 21 op/s 2023-12-12T10:00:45.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:44 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:00:45.311 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:45.312 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:45.510 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:45.510 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:45.526 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:45.526 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:45.526 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 89 -lt 100 ']' 2023-12-12T10:00:45.526 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 89 + 1 2023-12-12T10:00:45.567 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=90 2023-12-12T10:00:45.567 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:45.591 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:45.591 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:45.591 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:45.592 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:45.592 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:46.278 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:46.278 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:46.536 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:46.537 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:46.550 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:46.550 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:46.550 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 90 -lt 100 ']' 2023-12-12T10:00:46.550 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 90 + 1 2023-12-12T10:00:46.625 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=91 2023-12-12T10:00:46.625 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:46.643 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:46.643 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:46.644 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:46.645 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:46.645 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:46 smithi042 ceph-mon[98894]: pgmap v67: 129 pgs: 129 active+clean; 2.6 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-12-12T10:00:46.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:46 smithi111 ceph-mon[102565]: pgmap v67: 129 pgs: 129 active+clean; 2.6 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-12-12T10:00:47.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:46 smithi173 ceph-mon[103308]: pgmap v67: 129 pgs: 129 active+clean; 2.6 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-12-12T10:00:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 -- 172.21.15.42:0/735101549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181075a0 msgr2=0x7f3418107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 --2- 172.21.15.42:0/735101549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181075a0 0x7f3418107980 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f340c0099c0 tx=0x7f340c009cd0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 -- 172.21.15.42:0/735101549 shutdown_connections 2023-12-12T10:00:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 --2- 172.21.15.42:0/735101549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3418101ea0 0x7f341810e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 --2- 172.21.15.42:0/735101549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181075a0 0x7f3418107980 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 --2- 172.21.15.42:0/735101549 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f34181014e0 0x7f3418101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 -- 172.21.15.42:0/735101549 >> 172.21.15.42:0/735101549 conn(0x7f34180fcfd0 msgr2=0x7f34180ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 -- 172.21.15.42:0/735101549 shutdown_connections 2023-12-12T10:00:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 -- 172.21.15.42:0/735101549 wait complete. 2023-12-12T10:00:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.458+0000 7f3420660700 1 Processor -- start 2023-12-12T10:00:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f3420660700 1 -- start start 2023-12-12T10:00:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f3420660700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181014e0 0x7f341819d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f3420660700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418101ea0 0x7f341819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f3420660700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34181075a0 0x7f34181977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f3420660700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418197e40 con 0x7f34181014e0 2023-12-12T10:00:47.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f3420660700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418197fb0 con 0x7f3418101ea0 2023-12-12T10:00:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f3420660700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3418198170 con 0x7f34181075a0 2023-12-12T10:00:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341e3fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181014e0 0x7f341819d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341e3fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181014e0 0x7f341819d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36184/0 (socket says 172.21.15.42:36184) 2023-12-12T10:00:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341e3fc700 1 -- 172.21.15.42:0/4199068882 learned_addr learned my addr 172.21.15.42:0/4199068882 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:47.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341dbfb700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418101ea0 0x7f341819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341ebfd700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34181075a0 0x7f34181977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341e3fc700 1 -- 172.21.15.42:0/4199068882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34181075a0 msgr2=0x7f34181977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341e3fc700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34181075a0 0x7f34181977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341e3fc700 1 -- 172.21.15.42:0/4199068882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418101ea0 msgr2=0x7f341819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341e3fc700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418101ea0 0x7f341819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341e3fc700 1 -- 172.21.15.42:0/4199068882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0095f0 con 0x7f34181014e0 2023-12-12T10:00:47.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341dbfb700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418101ea0 0x7f341819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:00:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.459+0000 7f341e3fc700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181014e0 0x7f341819d6a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f340c00b580 tx=0x7f340c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.460+0000 7f340b7fe700 1 -- 172.21.15.42:0/4199068882 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c01c070 con 0x7f34181014e0 2023-12-12T10:00:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.460+0000 7f340b7fe700 1 -- 172.21.15.42:0/4199068882 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f340c00fe30 con 0x7f34181014e0 2023-12-12T10:00:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.460+0000 7f3420660700 1 -- 172.21.15.42:0/4199068882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34181983f0 con 0x7f34181014e0 2023-12-12T10:00:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.460+0000 7f340b7fe700 1 -- 172.21.15.42:0/4199068882 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c022d40 con 0x7f34181014e0 2023-12-12T10:00:47.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.460+0000 7f3420660700 1 -- 172.21.15.42:0/4199068882 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34181a4690 con 0x7f34181014e0 2023-12-12T10:00:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.462+0000 7f340b7fe700 1 -- 172.21.15.42:0/4199068882 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f340c020070 con 0x7f34181014e0 2023-12-12T10:00:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.463+0000 7f340b7fe700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34040a15f0 0x7f34040a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.463+0000 7f340b7fe700 1 -- 172.21.15.42:0/4199068882 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f340c0ce250 con 0x7f34181014e0 2023-12-12T10:00:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.463+0000 7f341dbfb700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34040a15f0 0x7f34040a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.463+0000 7f341dbfb700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34040a15f0 0x7f34040a3ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3418198bf0 tx=0x7f3414009040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:47.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.905+0000 7f342c859b80 1 Processor -- start 2023-12-12T10:00:47.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f342c859b80 1 -- start start 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f342c859b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x557191959580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f342c859b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x557191953e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f342c859b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 0x55719195ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f342c859b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571918aaaf0 con 0x557191953990 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f342c859b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571918a8410 con 0x5571919591a0 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f342c859b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571918a3d00 con 0x557191954350 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f341dbfb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x557191953e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f341dbfb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x557191953e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36194/0 (socket says 172.21.15.42:36194) 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.906+0000 7f341dbfb700 1 -- 172.21.15.42:0/2370095915 learned_addr learned my addr 172.21.15.42:0/2370095915 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f341e3fc700 1 --2- 172.21.15.42:0/2370095915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x557191959580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f341ebfd700 1 --2- 172.21.15.42:0/2370095915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 0x55719195ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f341dbfb700 1 -- 172.21.15.42:0/2370095915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 msgr2=0x55719195ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f341dbfb700 1 --2- 172.21.15.42:0/2370095915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 0x55719195ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f341dbfb700 1 -- 172.21.15.42:0/2370095915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 msgr2=0x557191959580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f341dbfb700 1 --2- 172.21.15.42:0/2370095915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x557191959580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f341dbfb700 1 -- 172.21.15.42:0/2370095915 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571917205d0 con 0x557191953990 2023-12-12T10:00:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f341dbfb700 1 --2- 172.21.15.42:0/2370095915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x557191953e10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3414016550 tx=0x7f3414012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34adb0ff1bc8600b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f3408ff9700 1 -- 172.21.15.42:0/2370095915 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414021600 con 0x557191953990 2023-12-12T10:00:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.907+0000 7f3408ff9700 1 -- 172.21.15.42:0/2370095915 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3414021c20 con 0x557191953990 2023-12-12T10:00:47.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.908+0000 7f3408ff9700 1 -- 172.21.15.42:0/2370095915 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414014bd0 con 0x557191953990 2023-12-12T10:00:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.908+0000 7f342c859b80 1 -- 172.21.15.42:0/2370095915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 msgr2=0x557191953e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.908+0000 7f342c859b80 1 --2- 172.21.15.42:0/2370095915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x557191953e10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3414016550 tx=0x7f3414012000 comp rx=0 tx=0).stop 2023-12-12T10:00:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.908+0000 7f342c859b80 1 -- 172.21.15.42:0/2370095915 shutdown_connections 2023-12-12T10:00:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.908+0000 7f342c859b80 1 --2- 172.21.15.42:0/2370095915 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x557191959580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.908+0000 7f342c859b80 1 --2- 172.21.15.42:0/2370095915 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 0x55719195ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.908+0000 7f342c859b80 1 --2- 172.21.15.42:0/2370095915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x557191953e10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.908+0000 7f342c859b80 1 -- 172.21.15.42:0/2370095915 >> 172.21.15.42:0/2370095915 conn(0x55719187e780 msgr2=0x557191949c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.909+0000 7f342c859b80 1 -- 172.21.15.42:0/2370095915 shutdown_connections 2023-12-12T10:00:47.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.909+0000 7f342c859b80 1 -- 172.21.15.42:0/2370095915 wait complete. 2023-12-12T10:00:47.945 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:47.945 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.950+0000 7f342c859b80 1 -- start start 2023-12-12T10:00:47.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.950+0000 7f342c859b80 1 Processor -- start 2023-12-12T10:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.950+0000 7f342c859b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x55719195fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.950+0000 7f342c859b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 0x55719195d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f342c859b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x55719195d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f342c859b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55719185d500 con 0x557191953990 2023-12-12T10:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f342c859b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55719185d1a0 con 0x5571919591a0 2023-12-12T10:00:47.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f342c859b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571917228c0 con 0x557191954350 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f342c859b80 10 client.0 _ll_register_callbacks cb 0x557191788a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341e3fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x55719195fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341e3fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x55719195fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36196/0 (socket says 172.21.15.42:36196) 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341e3fc700 1 -- 172.21.15.42:0/1950414608 learned_addr learned my addr 172.21.15.42:0/1950414608 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341dbfb700 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 0x55719195d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341ebfd700 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x55719195d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341e3fc700 1 -- 172.21.15.42:0/1950414608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 msgr2=0x55719195d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341e3fc700 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 0x55719195d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341e3fc700 1 -- 172.21.15.42:0/1950414608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 msgr2=0x55719195d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341e3fc700 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x55719195d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341e3fc700 1 -- 172.21.15.42:0/1950414608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3414007e00 con 0x557191953990 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f341e3fc700 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x55719195fec0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f340c000c00 tx=0x7f340c09abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623bd00400bdda2a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.951+0000 7f3401ffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:00:47.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.952+0000 7f3401ffb700 1 -- 172.21.15.42:0/1950414608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c09b490 con 0x557191953990 2023-12-12T10:00:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.952+0000 7f3401ffb700 1 -- 172.21.15.42:0/1950414608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f340c09b730 con 0x557191953990 2023-12-12T10:00:47.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.952+0000 7f342c859b80 1 -- 172.21.15.42:0/1950414608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557191942730 con 0x557191953990 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f3401ffb700 1 -- 172.21.15.42:0/1950414608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f340c022630 con 0x557191953990 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f342c859b80 10 client.15762 fetch_fsmap learned FSMap version 27 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f342c859b80 1 -- 172.21.15.42:0/1950414608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5571916ccd80 con 0x557191953990 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f3401ffb700 1 -- 172.21.15.42:0/1950414608 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f340c064080 con 0x557191953990 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f3401ffb700 1 -- 172.21.15.42:0/1950414608 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f340c09a520 con 0x557191953990 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f342c859b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f342c859b80 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f342c859b80 20 client.15762 populate_metadata read hostname 'smithi042' 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f342c859b80 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55719196d2b0 0x55719196f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f342c859b80 4 client.15762 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:00:47.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.963+0000 7f342c859b80 1 -- 172.21.15.42:0/1950414608 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55719191eee0 con 0x55719196d2b0 2023-12-12T10:00:47.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.964+0000 7f341dbfb700 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55719196d2b0 0x55719196f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.964+0000 7f3401ffb700 10 client.15762 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.964+0000 7f341dbfb700 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55719196d2b0 0x55719196f6a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1f518dffc972831 server_cookie=2cbdb8d4046c9a79 in_seq=0 out_seq=0 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f3401ffb700 1 -- 172.21.15.42:0/1950414608 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55719191eee0 con 0x55719196d2b0 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f3401ffb700 10 client.15762 handle_command_reply: tid=0 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 client.15762 shutdown 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 -- 172.21.15.42:0/1950414608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 msgr2=0x55719195fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x55719195fec0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f340c000c00 tx=0x7f340c09abf0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 -- 172.21.15.42:0/1950414608 shutdown_connections 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55719196d2b0 0x55719196f6a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x55719195d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191953990 0x55719195fec0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 --2- 172.21.15.42:0/1950414608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191954350 0x55719195d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f3401ffb700 0 client.15762 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 -- 172.21.15.42:0/1950414608 >> 172.21.15.42:0/1950414608 conn(0x55719187e780 msgr2=0x55719194e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 -- 172.21.15.42:0/1950414608 shutdown_connections 2023-12-12T10:00:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.965+0000 7f342c859b80 1 -- 172.21.15.42:0/1950414608 wait complete. 2023-12-12T10:00:47.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.966+0000 7f342c859b80 20 client.15762 trim_cache size 0 max 16384 2023-12-12T10:00:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.983+0000 7f342c859b80 1 Processor -- start 2023-12-12T10:00:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.983+0000 7f342c859b80 1 -- start start 2023-12-12T10:00:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f342c859b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 0x55719195d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f342c859b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557191954350 0x557191953dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f342c859b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5571919591a0 0x557191959580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f342c859b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55719196d0c0 con 0x5571919591a0 2023-12-12T10:00:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f342c859b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55719194a8b0 con 0x557191954350 2023-12-12T10:00:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f342c859b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557191974830 con 0x557191953990 2023-12-12T10:00:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341ebfd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5571919591a0 0x557191959580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341dbfb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557191954350 0x557191953dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341dbfb700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557191954350 0x557191953dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39264/0 (socket says 172.21.15.42:39264) 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341ebfd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5571919591a0 0x557191959580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36206/0 (socket says 172.21.15.42:36206) 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341dbfb700 1 -- 172.21.15.42:0/2152857032 learned_addr learned my addr 172.21.15.42:0/2152857032 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341e3fc700 1 --2- 172.21.15.42:0/2152857032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 0x55719195d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341dbfb700 1 -- 172.21.15.42:0/2152857032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 msgr2=0x55719195d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341dbfb700 1 --2- 172.21.15.42:0/2152857032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 0x55719195d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341dbfb700 1 -- 172.21.15.42:0/2152857032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5571919591a0 msgr2=0x557191959580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341dbfb700 1 --2- 172.21.15.42:0/2152857032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5571919591a0 0x557191959580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.984+0000 7f341dbfb700 1 -- 172.21.15.42:0/2152857032 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571916b5c00 con 0x557191954350 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.985+0000 7f341dbfb700 1 --2- 172.21.15.42:0/2152857032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557191954350 0x557191953dd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55719194c0f0 tx=0x7f3414014770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca847555d2580b6f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.985+0000 7f34097fa700 1 -- 172.21.15.42:0/2152857032 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414031070 con 0x557191954350 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.985+0000 7f34097fa700 1 -- 172.21.15.42:0/2152857032 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3414036070 con 0x557191954350 2023-12-12T10:00:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.985+0000 7f34097fa700 1 -- 172.21.15.42:0/2152857032 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414003c60 con 0x557191954350 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 -- 172.21.15.42:0/2152857032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557191954350 msgr2=0x557191953dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 --2- 172.21.15.42:0/2152857032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557191954350 0x557191953dd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x55719194c0f0 tx=0x7f3414014770 comp rx=0 tx=0).stop 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 -- 172.21.15.42:0/2152857032 shutdown_connections 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 --2- 172.21.15.42:0/2152857032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5571919591a0 0x557191959580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 --2- 172.21.15.42:0/2152857032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 0x55719195d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 --2- 172.21.15.42:0/2152857032 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557191954350 0x557191953dd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 -- 172.21.15.42:0/2152857032 >> 172.21.15.42:0/2152857032 conn(0x55719187e780 msgr2=0x557191949b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 -- 172.21.15.42:0/2152857032 shutdown_connections 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 -- 172.21.15.42:0/2152857032 wait complete. 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.986+0000 7f342c859b80 1 -- start start 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f342c859b80 1 Processor -- start 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f342c859b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 0x55719195bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f342c859b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191954350 0x55719195c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f342c859b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x5571919c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f342c859b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571919c2c60 con 0x557191954350 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f342c859b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571919c2da0 con 0x5571919591a0 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f342c859b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571919c2ee0 con 0x557191953990 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f342c859b80 10 client.0 _ll_register_callbacks cb 0x557191788a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f341dbfb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191954350 0x55719195c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f341dbfb700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191954350 0x55719195c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36210/0 (socket says 172.21.15.42:36210) 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.987+0000 7f341dbfb700 1 -- 172.21.15.42:0/2227771612 learned_addr learned my addr 172.21.15.42:0/2227771612 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f341ebfd700 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x5571919c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f341e3fc700 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 0x55719195bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f341dbfb700 1 -- 172.21.15.42:0/2227771612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 msgr2=0x55719195bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f341dbfb700 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 0x55719195bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f341dbfb700 1 -- 172.21.15.42:0/2227771612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 msgr2=0x5571919c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f341dbfb700 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x5571919c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f341dbfb700 1 -- 172.21.15.42:0/2227771612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571917205d0 con 0x557191954350 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f341ebfd700 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x5571919c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f341dbfb700 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191954350 0x55719195c120 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3414009c20 tx=0x7f34140095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc7dedc76d32d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f3401ffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f3401ffb700 1 -- 172.21.15.42:0/2227771612 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3414031070 con 0x557191954350 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f3401ffb700 1 -- 172.21.15.42:0/2227771612 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f34140144d0 con 0x557191954350 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f342c859b80 1 -- 172.21.15.42:0/2227771612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3414021c20 con 0x557191954350 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f3401ffb700 1 -- 172.21.15.42:0/2227771612 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3414003a00 con 0x557191954350 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f342c859b80 10 client.15768 fetch_fsmap learned FSMap version 27 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.988+0000 7f342c859b80 1 -- 172.21.15.42:0/2227771612 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5571916f31f0 con 0x557191954350 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.989+0000 7f3401ffb700 1 -- 172.21.15.42:0/2227771612 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3414021c20 con 0x557191954350 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.989+0000 7f3401ffb700 1 -- 172.21.15.42:0/2227771612 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f3414014d50 con 0x557191954350 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.989+0000 7f342c859b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.989+0000 7f342c859b80 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.989+0000 7f342c859b80 20 client.15768 populate_metadata read hostname 'smithi042' 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.989+0000 7f342c859b80 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5571919c4380 0x5571919c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:00:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.989+0000 7f342c859b80 4 client.15768 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:00:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.989+0000 7f342c859b80 1 -- 172.21.15.42:0/2227771612 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557191869f70 con 0x5571919c4380 2023-12-12T10:00:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.989+0000 7f341e3fc700 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5571919c4380 0x5571919c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:00:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.990+0000 7f3401ffb700 10 client.15768 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:47.990+0000 7f341e3fc700 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5571919c4380 0x5571919c67a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99896645b10fcb45 server_cookie=c71246cf6588032b in_seq=0 out_seq=0 2023-12-12T10:00:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f3401ffb700 1 -- 172.21.15.42:0/2227771612 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557191869f70 con 0x5571919c4380 2023-12-12T10:00:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f3401ffb700 10 client.15768 handle_command_reply: tid=0 2023-12-12T10:00:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 client.15768 shutdown 2023-12-12T10:00:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 -- 172.21.15.42:0/2227771612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191954350 msgr2=0x55719195c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191954350 0x55719195c120 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3414009c20 tx=0x7f34140095c0 comp rx=0 tx=0).stop 2023-12-12T10:00:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 -- 172.21.15.42:0/2227771612 shutdown_connections 2023-12-12T10:00:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5571919c4380 0x5571919c67a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:48.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5571919591a0 0x5571919c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557191953990 0x55719195bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f3401ffb700 0 client.15768 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 --2- 172.21.15.42:0/2227771612 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557191954350 0x55719195c120 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 -- 172.21.15.42:0/2227771612 >> 172.21.15.42:0/2227771612 conn(0x55719187e780 msgr2=0x55719195fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.007+0000 7f342c859b80 1 -- 172.21.15.42:0/2227771612 shutdown_connections 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.008+0000 7f342c859b80 1 -- 172.21.15.42:0/2227771612 wait complete. 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.008+0000 7f342c859b80 20 client.15768 trim_cache size 0 max 16384 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:00:48.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:00:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.008+0000 7f3420660700 1 -- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34040a15f0 msgr2=0x7f34040a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.008+0000 7f3420660700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34040a15f0 0x7f34040a3ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3418198bf0 tx=0x7f3414009040 comp rx=0 tx=0).stop 2023-12-12T10:00:48.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.008+0000 7f3420660700 1 -- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181014e0 msgr2=0x7f341819d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:00:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.008+0000 7f3420660700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181014e0 0x7f341819d6a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f340c00b580 tx=0x7f340c00f690 comp rx=0 tx=0).stop 2023-12-12T10:00:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.009+0000 7f3420660700 1 -- 172.21.15.42:0/4199068882 shutdown_connections 2023-12-12T10:00:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.009+0000 7f3420660700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f34181075a0 0x7f34181977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.009+0000 7f3420660700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f34181014e0 0x7f341819d6a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.009+0000 7f3420660700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f34040a15f0 0x7f34040a3ab0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.009+0000 7f3420660700 1 --2- 172.21.15.42:0/4199068882 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3418101ea0 0x7f341819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:00:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.009+0000 7f3420660700 1 -- 172.21.15.42:0/4199068882 >> 172.21.15.42:0/4199068882 conn(0x7f34180fcfd0 msgr2=0x7f34180fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:00:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.009+0000 7f3420660700 1 -- 172.21.15.42:0/4199068882 shutdown_connections 2023-12-12T10:00:48.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:00:48.009+0000 7f3420660700 1 -- 172.21.15.42:0/4199068882 wait complete. 2023-12-12T10:00:48.132 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:48.132 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:48.163 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:48.163 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:48.163 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 91 -lt 100 ']' 2023-12-12T10:00:48.163 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 91 + 1 2023-12-12T10:00:48.200 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=92 2023-12-12T10:00:48.200 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:48.224 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:48.224 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:48.225 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:48.225 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:48.226 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:48.903 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:48.903 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:48 smithi042 ceph-mon[98894]: pgmap v68: 129 pgs: 129 active+clean; 2.6 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-12-12T10:00:48.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:48 smithi111 ceph-mon[102565]: pgmap v68: 129 pgs: 129 active+clean; 2.6 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-12-12T10:00:49.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:48 smithi173 ceph-mon[103308]: pgmap v68: 129 pgs: 129 active+clean; 2.6 MiB data, 336 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 11 op/s 2023-12-12T10:00:49.092 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:49.092 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:49.138 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:49.138 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:49.138 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 92 -lt 100 ']' 2023-12-12T10:00:49.138 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 92 + 1 2023-12-12T10:00:49.192 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=93 2023-12-12T10:00:49.192 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:49.225 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:49.225 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:49.225 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:49.226 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:49.226 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:49.977 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:49.978 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:50.151 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:50.151 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:50.164 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:50.164 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:50.164 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 93 -lt 100 ']' 2023-12-12T10:00:50.164 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 93 + 1 2023-12-12T10:00:50.199 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=94 2023-12-12T10:00:50.200 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:50.207 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:50.207 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:50.208 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:50.208 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:50.208 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:50.644 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:50.644 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:50.851 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:50.851 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:50.872 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:50.872 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:50.872 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 94 -lt 100 ']' 2023-12-12T10:00:50.872 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 94 + 1 2023-12-12T10:00:50.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:50 smithi042 ceph-mon[98894]: pgmap v69: 129 pgs: 129 active+clean; 2.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 16 op/s 2023-12-12T10:00:50.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:50 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:00:50.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:50 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:00:50.908 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=95 2023-12-12T10:00:50.908 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:50.943 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:50.943 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:50.943 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:50.944 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:50.944 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:50 smithi111 ceph-mon[102565]: pgmap v69: 129 pgs: 129 active+clean; 2.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 16 op/s 2023-12-12T10:00:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:50 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:00:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:50 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:00:51.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:50 smithi173 ceph-mon[103308]: pgmap v69: 129 pgs: 129 active+clean; 2.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 KiB/s wr, 16 op/s 2023-12-12T10:00:51.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:50 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:00:51.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:50 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:00:51.627 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:51.627 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:51.767 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:51.768 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:51.796 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:51.796 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:51.796 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 95 -lt 100 ']' 2023-12-12T10:00:51.796 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 95 + 1 2023-12-12T10:00:51.824 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=96 2023-12-12T10:00:51.824 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:51.882 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:51.883 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:51.883 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:51.883 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:51.884 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:52.777 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:52.777 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:52.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:52 smithi042 ceph-mon[98894]: pgmap v70: 129 pgs: 129 active+clean; 2.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-12-12T10:00:52.944 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:52.944 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:52.957 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:52.957 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:52.957 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 96 -lt 100 ']' 2023-12-12T10:00:52.957 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 96 + 1 2023-12-12T10:00:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:52 smithi111 ceph-mon[102565]: pgmap v70: 129 pgs: 129 active+clean; 2.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-12-12T10:00:53.003 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=97 2023-12-12T10:00:53.003 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:52 smithi173 ceph-mon[103308]: pgmap v70: 129 pgs: 129 active+clean; 2.7 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 10 op/s 2023-12-12T10:00:53.090 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:53.090 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:53.091 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:53.091 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:53.091 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:53.727 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:53.727 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:53.917 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:53.918 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:53.937 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:53.937 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:53.937 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 97 -lt 100 ']' 2023-12-12T10:00:53.938 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 97 + 1 2023-12-12T10:00:53.966 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=98 2023-12-12T10:00:53.966 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:53.978 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:53.978 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:53.979 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:53.979 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:53.979 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:54.518 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:54.519 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:54.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:54 smithi042 ceph-mon[98894]: pgmap v71: 129 pgs: 129 active+clean; 2.8 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 21 op/s 2023-12-12T10:00:54.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:54 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:00:54.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:54 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:00:54.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:54 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:00:54.684 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:54.684 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:54.704 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:54.704 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:54.704 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 98 -lt 100 ']' 2023-12-12T10:00:54.704 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 98 + 1 2023-12-12T10:00:54.757 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=99 2023-12-12T10:00:54.757 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:54.764 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:54.764 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:54.765 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:54.765 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:54.765 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:54 smithi111 ceph-mon[102565]: pgmap v71: 129 pgs: 129 active+clean; 2.8 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 21 op/s 2023-12-12T10:00:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:54 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:00:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:54 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:00:54.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:54 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:00:55.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:54 smithi173 ceph-mon[103308]: pgmap v71: 129 pgs: 129 active+clean; 2.8 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 21 op/s 2023-12-12T10:00:55.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:54 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:00:55.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:54 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:00:55.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:54 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:00:55.085 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:55.085 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:55.217 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:55.217 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:55.230 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:55.230 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:55.230 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 99 -lt 100 ']' 2023-12-12T10:00:55.230 INFO:tasks.workunit.client.0.smithi042.stderr:++ expr 99 + 1 2023-12-12T10:00:55.274 INFO:tasks.workunit.client.0.smithi042.stderr:+ c=100 2023-12-12T10:00:55.274 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:55.555 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:55.556 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:55.557 INFO:tasks.workunit.client.0.smithi042.stderr:+ c1=2 2023-12-12T10:00:55.557 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo 3 2023-12-12T10:00:55.557 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-12T10:00:56.010 INFO:tasks.workunit.client.0.smithi042.stderr:++ getfacl d1 2023-12-12T10:00:56.011 INFO:tasks.workunit.client.0.smithi042.stderr:++ grep -c nobody:rw 2023-12-12T10:00:56.150 INFO:tasks.workunit.client.0.smithi042.stderr:+ c2=2 2023-12-12T10:00:56.150 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:56.162 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:56.162 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 2 -ne 2 ']' 2023-12-12T10:00:56.162 INFO:tasks.workunit.client.0.smithi042.stderr:+ '[' 100 -lt 100 ']' 2023-12-12T10:00:56.167 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir d1 2023-12-12T10:00:56.172 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2023-12-12T10:00:56.186 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2023-12-12T10:00:56.187 INFO:tasks.workunit.client.0.smithi042.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2023-12-12T10:00:56.187 INFO:tasks.workunit.client.0.smithi042.stderr:+ getfattr -n system.posix_acl_access d1 2023-12-12T10:00:56.196 INFO:tasks.workunit.client.0.smithi042.stderr:d1: system.posix_acl_access: No such attribute 2023-12-12T10:00:56.196 INFO:tasks.workunit.client.0.smithi042.stderr:+ return 0 2023-12-12T10:00:56.196 INFO:tasks.workunit.client.0.smithi042.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2023-12-12T10:00:56.196 INFO:tasks.workunit.client.0.smithi042.stderr:+ getfattr -n system.posix_acl_default . 2023-12-12T10:00:56.197 INFO:tasks.workunit.client.0.smithi042.stderr:.: system.posix_acl_default: No such attribute 2023-12-12T10:00:56.197 INFO:tasks.workunit.client.0.smithi042.stderr:+ return 0 2023-12-12T10:00:56.197 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir d1 2023-12-12T10:00:56.199 INFO:tasks.workunit.client.0.smithi042.stderr:+ cd .. 2023-12-12T10:00:56.199 INFO:tasks.workunit.client.0.smithi042.stderr:+ rmdir testdir 2023-12-12T10:00:56.215 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo OK 2023-12-12T10:00:56.216 INFO:tasks.workunit.client.0.smithi042.stdout:OK 2023-12-12T10:00:56.216 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-12T10:00:56.217 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-12T10:00:56.581 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2023-12-12T10:00:56.581 DEBUG:teuthology.orchestra.run.smithi042:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2023-12-12T10:00:56.658 INFO:tasks.workunit.client.0.smithi042.stderr:+ set -e 2023-12-12T10:00:56.658 INFO:tasks.workunit.client.0.smithi042.stderr:+ file=test_chmod.171520 2023-12-12T10:00:56.658 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo foo 2023-12-12T10:00:56.662 INFO:tasks.workunit.client.0.smithi042.stderr:+ test 0 '!=' 0 2023-12-12T10:00:56.662 INFO:tasks.workunit.client.0.smithi042.stderr:+ chmod 400 test_chmod.171520 2023-12-12T10:00:56.663 INFO:tasks.workunit.client.0.smithi042.stderr:+ test 0 '!=' 0 2023-12-12T10:00:56.663 INFO:tasks.workunit.client.0.smithi042.stderr:+ check_perms test_chmod.171520 -r-------- 2023-12-12T10:00:56.663 INFO:tasks.workunit.client.0.smithi042.stderr:+ file=test_chmod.171520 2023-12-12T10:00:56.664 INFO:tasks.workunit.client.0.smithi042.stderr:++ ls -la test_chmod.171520 2023-12-12T10:00:56.666 INFO:tasks.workunit.client.0.smithi042.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Dec 12 10:00 test_chmod.171520' 2023-12-12T10:00:56.666 INFO:tasks.workunit.client.0.smithi042.stderr:+ test 0 '!=' 0 2023-12-12T10:00:56.666 INFO:tasks.workunit.client.0.smithi042.stderr:+ perms=-r-------- 2023-12-12T10:00:56.666 INFO:tasks.workunit.client.0.smithi042.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Dec 12 10:00 test_chmod.171520 2023-12-12T10:00:56.666 INFO:tasks.workunit.client.0.smithi042.stderr:++ awk '{print $1}' 2023-12-12T10:00:56.731 INFO:tasks.workunit.client.0.smithi042.stderr:+ test -r-------- '!=' -r--------. 2023-12-12T10:00:56.732 INFO:tasks.workunit.client.0.smithi042.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Dec 12 10:00 test_chmod.171520 2023-12-12T10:00:56.732 INFO:tasks.workunit.client.0.smithi042.stderr:++ awk '{print $1}' 2023-12-12T10:00:56.734 INFO:tasks.workunit.client.0.smithi042.stderr:+ test -r--------. '!=' -r--------. 2023-12-12T10:00:56.734 INFO:tasks.workunit.client.0.smithi042.stderr:+ set +e 2023-12-12T10:00:56.734 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo bar 2023-12-12T10:00:56.734 INFO:tasks.workunit.client.0.smithi042.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.171520: Permission denied 2023-12-12T10:00:56.734 INFO:tasks.workunit.client.0.smithi042.stderr:+ test 1 = 0 2023-12-12T10:00:56.734 INFO:tasks.workunit.client.0.smithi042.stderr:+ set -e 2023-12-12T10:00:56.734 INFO:tasks.workunit.client.0.smithi042.stderr:+ chmod 600 test_chmod.171520 2023-12-12T10:00:56.735 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo bar 2023-12-12T10:00:56.735 INFO:tasks.workunit.client.0.smithi042.stderr:+ test 0 '!=' 0 2023-12-12T10:00:56.735 INFO:tasks.workunit.client.0.smithi042.stderr:+ check_perms test_chmod.171520 -rw------- 2023-12-12T10:00:56.735 INFO:tasks.workunit.client.0.smithi042.stderr:+ file=test_chmod.171520 2023-12-12T10:00:56.735 INFO:tasks.workunit.client.0.smithi042.stderr:++ ls -la test_chmod.171520 2023-12-12T10:00:56.737 INFO:tasks.workunit.client.0.smithi042.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Dec 12 10:00 test_chmod.171520' 2023-12-12T10:00:56.737 INFO:tasks.workunit.client.0.smithi042.stderr:+ test 0 '!=' 0 2023-12-12T10:00:56.737 INFO:tasks.workunit.client.0.smithi042.stderr:+ perms=-rw------- 2023-12-12T10:00:56.737 INFO:tasks.workunit.client.0.smithi042.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Dec 12 10:00 test_chmod.171520 2023-12-12T10:00:56.737 INFO:tasks.workunit.client.0.smithi042.stderr:++ awk '{print $1}' 2023-12-12T10:00:56.738 INFO:tasks.workunit.client.0.smithi042.stderr:+ test -rw------- '!=' -rw-------. 2023-12-12T10:00:56.739 INFO:tasks.workunit.client.0.smithi042.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Dec 12 10:00 test_chmod.171520 2023-12-12T10:00:56.739 INFO:tasks.workunit.client.0.smithi042.stderr:++ awk '{print $1}' 2023-12-12T10:00:56.741 INFO:tasks.workunit.client.0.smithi042.stderr:+ test -rw-------. '!=' -rw-------. 2023-12-12T10:00:56.741 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo foo 2023-12-12T10:00:56.741 INFO:tasks.workunit.client.0.smithi042.stderr:+ test 0 '!=' 0 2023-12-12T10:00:56.742 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-12T10:00:56.742 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-12T10:00:56.790 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2023-12-12T10:00:56.790 DEBUG:teuthology.orchestra.run.smithi042:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2023-12-12T10:00:56.815 INFO:tasks.workunit.client.0.smithi042.stderr:+ set -e 2023-12-12T10:00:56.816 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir -p testdir 2023-12-12T10:00:56.817 INFO:tasks.workunit.client.0.smithi042.stderr:+ file=test_chmod.171579 2023-12-12T10:00:56.818 INFO:tasks.workunit.client.0.smithi042.stderr:+ echo foo 2023-12-12T10:00:56.818 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo chmod 600 testdir 2023-12-12T10:00:56.827 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:56 smithi042 ceph-mon[98894]: pgmap v72: 129 pgs: 129 active+clean; 2.8 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-12-12T10:00:56.838 INFO:tasks.workunit.client.0.smithi042.stderr:+ expect_failure cat testdir/test_chmod.171579 2023-12-12T10:00:56.838 INFO:tasks.workunit.client.0.smithi042.stderr:+ cat testdir/test_chmod.171579 2023-12-12T10:00:56.844 INFO:tasks.workunit.client.0.smithi042.stderr:cat: testdir/test_chmod.171579: Permission denied 2023-12-12T10:00:56.844 INFO:tasks.workunit.client.0.smithi042.stderr:+ return 0 2023-12-12T10:00:56.844 INFO:tasks.workunit.client.0.smithi042.stderr:+ sudo cat testdir/test_chmod.171579 2023-12-12T10:00:56.854 INFO:tasks.workunit.client.0.smithi042.stdout:foo 2023-12-12T10:00:56.856 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-12T10:00:56.856 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-12T10:00:56.946 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2023-12-12T10:00:56.946 DEBUG:teuthology.orchestra.run.smithi042:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2023-12-12T10:00:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:56 smithi111 ceph-mon[102565]: pgmap v72: 129 pgs: 129 active+clean; 2.8 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-12-12T10:00:57.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:56 smithi173 ceph-mon[103308]: pgmap v72: 129 pgs: 129 active+clean; 2.8 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-12-12T10:00:57.818 INFO:tasks.workunit.client.0.smithi042.stdout:ok 2023-12-12T10:00:57.821 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-12T10:00:57.821 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-12T10:00:57.865 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2023-12-12T10:00:57.866 DEBUG:teuthology.orchestra.run.smithi042:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2023-12-12T10:00:57.951 INFO:tasks.workunit.client.0.smithi042.stdout:creating folder hierarchy 2023-12-12T10:00:58.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:00:58 smithi042 ceph-mon[98894]: pgmap v73: 129 pgs: 129 active+clean; 2.8 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-12-12T10:00:58.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:00:58 smithi111 ceph-mon[102565]: pgmap v73: 129 pgs: 129 active+clean; 2.8 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-12-12T10:00:59.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:00:58 smithi173 ceph-mon[103308]: pgmap v73: 129 pgs: 129 active+clean; 2.8 MiB data, 337 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2023-12-12T10:01:00.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:00 smithi042 ceph-mon[98894]: pgmap v74: 129 pgs: 129 active+clean; 3.0 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 22 op/s 2023-12-12T10:01:00.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:00 smithi111 ceph-mon[102565]: pgmap v74: 129 pgs: 129 active+clean; 3.0 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 22 op/s 2023-12-12T10:01:01.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:00 smithi173 ceph-mon[103308]: pgmap v74: 129 pgs: 129 active+clean; 3.0 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 38 KiB/s wr, 22 op/s 2023-12-12T10:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.630+0000 7f0480181700 1 -- 172.21.15.42:0/367864134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106a40 msgr2=0x7f047810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.630+0000 7f0480181700 1 --2- 172.21.15.42:0/367864134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106a40 0x7f047810e320 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f047000b0a0 tx=0x7f047000b3b0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.630+0000 7f0480181700 1 -- 172.21.15.42:0/367864134 shutdown_connections 2023-12-12T10:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.630+0000 7f0480181700 1 --2- 172.21.15.42:0/367864134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106a40 0x7f047810e320 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.630+0000 7f0480181700 1 --2- 172.21.15.42:0/367864134 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0478068ca0 0x7f0478069080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.630+0000 7f0480181700 1 --2- 172.21.15.42:0/367864134 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0478106080 0x7f0478106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.630+0000 7f0480181700 1 -- 172.21.15.42:0/367864134 >> 172.21.15.42:0/367864134 conn(0x7f0478077570 msgr2=0x7f0478077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.630+0000 7f0480181700 1 -- 172.21.15.42:0/367864134 shutdown_connections 2023-12-12T10:01:01.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.631+0000 7f0480181700 1 -- 172.21.15.42:0/367864134 wait complete. 2023-12-12T10:01:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.631+0000 7f0480181700 1 Processor -- start 2023-12-12T10:01:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.631+0000 7f0480181700 1 -- start start 2023-12-12T10:01:01.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.631+0000 7f0480181700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0478068ca0 0x7f047819d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f0480181700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106080 0x7f047819dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f0480181700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0478106a40 0x7f0478197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f0480181700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0478197f80 con 0x7f0478106a40 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f0480181700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04781980f0 con 0x7f0478106080 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f0480181700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04781982b0 con 0x7f0478068ca0 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047d71c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106080 0x7f047819dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047e71e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0478106a40 0x7f0478197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047df1d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0478068ca0 0x7f047819d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047d71c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106080 0x7f047819dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39328/0 (socket says 172.21.15.42:39328) 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047d71c700 1 -- 172.21.15.42:0/3735465260 learned_addr learned my addr 172.21.15.42:0/3735465260 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047d71c700 1 -- 172.21.15.42:0/3735465260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0478068ca0 msgr2=0x7f047819d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047d71c700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0478068ca0 0x7f047819d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047d71c700 1 -- 172.21.15.42:0/3735465260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0478106a40 msgr2=0x7f0478197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047d71c700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0478106a40 0x7f0478197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047d71c700 1 -- 172.21.15.42:0/3735465260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0468009520 con 0x7f0478106080 2023-12-12T10:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f047d71c700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106080 0x7f047819dd30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f046800eb30 tx=0x7f046800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f046effd700 1 -- 172.21.15.42:0/3735465260 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468009880 con 0x7f0478106080 2023-12-12T10:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f046effd700 1 -- 172.21.15.42:0/3735465260 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f04680044e0 con 0x7f0478106080 2023-12-12T10:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.632+0000 7f046effd700 1 -- 172.21.15.42:0/3735465260 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0468003d80 con 0x7f0478106080 2023-12-12T10:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.633+0000 7f0480181700 1 -- 172.21.15.42:0/3735465260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0470009d00 con 0x7f0478106080 2023-12-12T10:01:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.633+0000 7f0480181700 1 -- 172.21.15.42:0/3735465260 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04781a4600 con 0x7f0478106080 2023-12-12T10:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.635+0000 7f046effd700 1 -- 172.21.15.42:0/3735465260 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f0468003f20 con 0x7f0478106080 2023-12-12T10:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.635+0000 7f046effd700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f04640a15f0 0x7f04640a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.636+0000 7f047df1d700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f04640a15f0 0x7f04640a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.636+0000 7f046effd700 1 -- 172.21.15.42:0/3735465260 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0468014070 con 0x7f0478106080 2023-12-12T10:01:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.636+0000 7f047df1d700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f04640a15f0 0x7f04640a3ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0474005950 tx=0x7f047400b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.670+0000 7f048c37ab80 1 Processor -- start 2023-12-12T10:01:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f048c37ab80 1 -- start start 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 0x562614f88530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f82990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f82ed0 0x562614f83310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f048c37ab80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614ed9af0 con 0x562614f88a70 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f048c37ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614ed7410 con 0x562614f82ed0 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f048c37ab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614ed2d00 con 0x562614f88150 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f047d71c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f82990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f047d71c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f82990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36278/0 (socket says 172.21.15.42:36278) 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f047d71c700 1 -- 172.21.15.42:0/756119082 learned_addr learned my addr 172.21.15.42:0/756119082 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f047e71e700 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f82ed0 0x562614f83310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f047df1d700 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 0x562614f88530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f047d71c700 1 -- 172.21.15.42:0/756119082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 msgr2=0x562614f88530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f047d71c700 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 0x562614f88530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f047d71c700 1 -- 172.21.15.42:0/756119082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f82ed0 msgr2=0x562614f83310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.671+0000 7f047d71c700 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f82ed0 0x562614f83310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.672+0000 7f047d71c700 1 -- 172.21.15.42:0/756119082 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562614d4f5d0 con 0x562614f88a70 2023-12-12T10:01:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.672+0000 7f047df1d700 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 0x562614f88530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:01:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.672+0000 7f047d71c700 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f82990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0468009230 tx=0x7f0468099f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c9dbb78dcc36c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.672+0000 7f0463fff700 1 -- 172.21.15.42:0/756119082 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046809a5b0 con 0x562614f88a70 2023-12-12T10:01:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.672+0000 7f0463fff700 1 -- 172.21.15.42:0/756119082 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f046809e700 con 0x562614f88a70 2023-12-12T10:01:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.672+0000 7f0463fff700 1 -- 172.21.15.42:0/756119082 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046801fec0 con 0x562614f88a70 2023-12-12T10:01:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 -- 172.21.15.42:0/756119082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 msgr2=0x562614f82990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f82990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0468009230 tx=0x7f0468099f10 comp rx=0 tx=0).stop 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 -- 172.21.15.42:0/756119082 shutdown_connections 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f82ed0 0x562614f83310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 0x562614f88530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 --2- 172.21.15.42:0/756119082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f82990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 -- 172.21.15.42:0/756119082 >> 172.21.15.42:0/756119082 conn(0x562614ead780 msgr2=0x562614f78c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 -- 172.21.15.42:0/756119082 shutdown_connections 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 -- 172.21.15.42:0/756119082 wait complete. 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 -- start start 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.673+0000 7f048c37ab80 1 Processor -- start 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f8eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 0x562614f8bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88a70 0x562614f8c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f048c37ab80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614e8c500 con 0x562614f82ed0 2023-12-12T10:01:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f048c37ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614e8c1a0 con 0x562614f88a70 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f048c37ab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614d518c0 con 0x562614f88150 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f048c37ab80 10 client.0 _ll_register_callbacks cb 0x562614db7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047df1d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f8eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047df1d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f8eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36282/0 (socket says 172.21.15.42:36282) 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047df1d700 1 -- 172.21.15.42:0/83302643 learned_addr learned my addr 172.21.15.42:0/83302643 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047e71e700 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88a70 0x562614f8c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047d71c700 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 0x562614f8bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047df1d700 1 -- 172.21.15.42:0/83302643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 msgr2=0x562614f8bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047df1d700 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 0x562614f8bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047df1d700 1 -- 172.21.15.42:0/83302643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88a70 msgr2=0x562614f8c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047df1d700 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88a70 0x562614f8c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047df1d700 1 -- 172.21.15.42:0/83302643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04781a4600 con 0x562614f82ed0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f047df1d700 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f8eab0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0474011540 tx=0x7f04740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e6667d490f3f8c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.674+0000 7f04617fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f04617fa700 1 -- 172.21.15.42:0/83302643 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047401f840 con 0x562614f82ed0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f04617fa700 1 -- 172.21.15.42:0/83302643 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0474017070 con 0x562614f82ed0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f048c37ab80 1 -- 172.21.15.42:0/83302643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562614f71730 con 0x562614f82ed0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f04617fa700 1 -- 172.21.15.42:0/83302643 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0474011e50 con 0x562614f82ed0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f048c37ab80 10 client.15798 fetch_fsmap learned FSMap version 27 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f048c37ab80 1 -- 172.21.15.42:0/83302643 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562614cfbd80 con 0x562614f82ed0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f04617fa700 1 -- 172.21.15.42:0/83302643 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562614f71730 con 0x562614f82ed0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f04617fa700 1 -- 172.21.15.42:0/83302643 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f0474028b30 con 0x562614f82ed0 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f048c37ab80 10 client.15798 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f048c37ab80 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f048c37ab80 20 client.15798 populate_metadata read hostname 'smithi042' 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f048c37ab80 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562614f9c510 0x562614f9e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f048c37ab80 4 client.15798 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.675+0000 7f048c37ab80 1 -- 172.21.15.42:0/83302643 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562614f4dee0 con 0x562614f9c510 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.676+0000 7f047d71c700 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562614f9c510 0x562614f9e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.676+0000 7f04617fa700 10 client.15798 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.676+0000 7f047d71c700 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562614f9c510 0x562614f9e930 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bcfe7284d639dec server_cookie=ae67ae2ef81d26ac in_seq=0 out_seq=0 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f04617fa700 1 -- 172.21.15.42:0/83302643 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562614f4dee0 con 0x562614f9c510 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f04617fa700 10 client.15798 handle_command_reply: tid=0 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 client.15798 shutdown 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 -- 172.21.15.42:0/83302643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 msgr2=0x562614f8eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f8eab0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0474011540 tx=0x7f04740047c0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 -- 172.21.15.42:0/83302643 shutdown_connections 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562614f9c510 0x562614f9e930 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88a70 0x562614f8c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f8eab0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 --2- 172.21.15.42:0/83302643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88150 0x562614f8bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f04617fa700 0 client.15798 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 -- 172.21.15.42:0/83302643 >> 172.21.15.42:0/83302643 conn(0x562614ead780 msgr2=0x562614f81980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.677+0000 7f048c37ab80 1 -- 172.21.15.42:0/83302643 shutdown_connections 2023-12-12T10:01:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.678+0000 7f048c37ab80 1 -- 172.21.15.42:0/83302643 wait complete. 2023-12-12T10:01:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.678+0000 7f048c37ab80 20 client.15798 trim_cache size 0 max 16384 2023-12-12T10:01:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.693+0000 7f048c37ab80 1 Processor -- start 2023-12-12T10:01:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.693+0000 7f048c37ab80 1 -- start start 2023-12-12T10:01:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.693+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f82ed0 0x562614f97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.693+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f82230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.693+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f7a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.693+0000 7f048c37ab80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614f8d3a0 con 0x562614f88a70 2023-12-12T10:01:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.693+0000 7f048c37ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614f71700 con 0x562614f88150 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.693+0000 7f048c37ab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614f755d0 con 0x562614f82ed0 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047d71c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f82230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047df1d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f82ed0 0x562614f97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047d71c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f82230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39352/0 (socket says 172.21.15.42:39352) 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047df1d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f82ed0 0x562614f97f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51274/0 (socket says 172.21.15.42:51274) 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047d71c700 1 -- 172.21.15.42:0/3539283936 learned_addr learned my addr 172.21.15.42:0/3539283936 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047e71e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f7a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047d71c700 1 -- 172.21.15.42:0/3539283936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f82ed0 msgr2=0x562614f97f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047d71c700 1 --2- 172.21.15.42:0/3539283936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f82ed0 0x562614f97f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047d71c700 1 -- 172.21.15.42:0/3539283936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 msgr2=0x562614f7a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047d71c700 1 --2- 172.21.15.42:0/3539283936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f7a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047d71c700 1 -- 172.21.15.42:0/3539283936 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562614ce4c00 con 0x562614f88150 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047e71e700 1 --2- 172.21.15.42:0/3539283936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f7a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:01:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f047d71c700 1 --2- 172.21.15.42:0/3539283936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f82230 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f046800eb00 tx=0x7f0468085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8dbe6adb69f273b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f046cff9700 1 -- 172.21.15.42:0/3539283936 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046809a5b0 con 0x562614f88150 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f046cff9700 1 -- 172.21.15.42:0/3539283936 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f046800cdd0 con 0x562614f88150 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.694+0000 7f046cff9700 1 -- 172.21.15.42:0/3539283936 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046809a810 con 0x562614f88150 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 -- 172.21.15.42:0/3539283936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 msgr2=0x562614f82230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3539283936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f82230 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f046800eb00 tx=0x7f0468085040 comp rx=0 tx=0).stop 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 -- 172.21.15.42:0/3539283936 shutdown_connections 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3539283936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f88a70 0x562614f7a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3539283936 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f82ed0 0x562614f97f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3539283936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f82230 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 -- 172.21.15.42:0/3539283936 >> 172.21.15.42:0/3539283936 conn(0x562614ead780 msgr2=0x562614f81980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 -- 172.21.15.42:0/3539283936 shutdown_connections 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 -- 172.21.15.42:0/3539283936 wait complete. 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.695+0000 7f048c37ab80 1 -- start start 2023-12-12T10:01:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f048c37ab80 1 Processor -- start 2023-12-12T10:01:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f91460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f048c37ab80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88a70 0x562614f91ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f048c37ab80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614ff19d0 con 0x562614f82ed0 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f048c37ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614ff1b10 con 0x562614f88150 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f048c37ab80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562614ff1c50 con 0x562614f88a70 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f048c37ab80 10 client.0 _ll_register_callbacks cb 0x562614db7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047df1d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f91460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047df1d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f91460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36296/0 (socket says 172.21.15.42:36296) 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047df1d700 1 -- 172.21.15.42:0/3674989219 learned_addr learned my addr 172.21.15.42:0/3674989219 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047d71c700 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047e71e700 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88a70 0x562614f91ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047df1d700 1 -- 172.21.15.42:0/3674989219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88a70 msgr2=0x562614f91ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047df1d700 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88a70 0x562614f91ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047df1d700 1 -- 172.21.15.42:0/3674989219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 msgr2=0x562614f919a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047df1d700 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f919a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.696+0000 7f047df1d700 1 -- 172.21.15.42:0/3674989219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047810ec20 con 0x562614f82ed0 2023-12-12T10:01:01.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f047df1d700 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f91460 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0474028ec0 tx=0x7f047401e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7e6d5c6c82a641 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f04617fa700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f04617fa700 1 -- 172.21.15.42:0/3674989219 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047401ed10 con 0x562614f82ed0 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f048c37ab80 1 -- 172.21.15.42:0/3674989219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f046809e700 con 0x562614f82ed0 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f04617fa700 1 -- 172.21.15.42:0/3674989219 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0474017070 con 0x562614f82ed0 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f04617fa700 1 -- 172.21.15.42:0/3674989219 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0474010a40 con 0x562614f82ed0 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f048c37ab80 10 client.15804 fetch_fsmap learned FSMap version 27 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f048c37ab80 1 -- 172.21.15.42:0/3674989219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562614d4f5d0 con 0x562614f82ed0 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f04617fa700 1 -- 172.21.15.42:0/3674989219 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f046809e700 con 0x562614f82ed0 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f04617fa700 1 -- 172.21.15.42:0/3674989219 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 27) v1 ==== 7890+0+0 (secure 0 0 0) 0x7f047403c060 con 0x562614f82ed0 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f048c37ab80 10 client.15804 fetch_fsmap finished waiting for FSMap version 27 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f048c37ab80 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f048c37ab80 20 client.15804 populate_metadata read hostname 'smithi042' 2023-12-12T10:01:01.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562614ff3320 0x562614ff5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f048c37ab80 4 client.15804 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:01:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.697+0000 7f048c37ab80 1 -- 172.21.15.42:0/3674989219 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562614e98f70 con 0x562614ff3320 2023-12-12T10:01:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.698+0000 7f047d71c700 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562614ff3320 0x562614ff5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.698+0000 7f04617fa700 10 client.15804 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:01.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:01.698+0000 7f047d71c700 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562614ff3320 0x562614ff5740 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a1f9ec41ba1509 server_cookie=6929f39cddcb7680 in_seq=0 out_seq=0 2023-12-12T10:01:02.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:02 smithi042 ceph-mon[98894]: pgmap v75: 129 pgs: 129 active+clean; 3.0 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 17 op/s 2023-12-12T10:01:02.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:02 smithi111 ceph-mon[102565]: pgmap v75: 129 pgs: 129 active+clean; 3.0 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 17 op/s 2023-12-12T10:01:03.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:02 smithi173 ceph-mon[103308]: pgmap v75: 129 pgs: 129 active+clean; 3.0 MiB data, 338 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 17 op/s 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f04617fa700 1 -- 172.21.15.42:0/3674989219 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562614e98f70 con 0x562614ff3320 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f04617fa700 10 client.15804 handle_command_reply: tid=0 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 client.15804 shutdown 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 -- 172.21.15.42:0/3674989219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 msgr2=0x562614f91460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f91460 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0474028ec0 tx=0x7f047401e840 comp rx=0 tx=0).stop 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 -- 172.21.15.42:0/3674989219 shutdown_connections 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562614f88a70 0x562614f91ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562614f82ed0 0x562614f91460 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562614ff3320 0x562614ff5740 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 --2- 172.21.15.42:0/3674989219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562614f88150 0x562614f919a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f04617fa700 0 client.15804 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 -- 172.21.15.42:0/3674989219 >> 172.21.15.42:0/3674989219 conn(0x562614ead780 msgr2=0x562614f8e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 -- 172.21.15.42:0/3674989219 shutdown_connections 2023-12-12T10:01:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 1 -- 172.21.15.42:0/3674989219 wait complete. 2023-12-12T10:01:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.372+0000 7f048c37ab80 20 client.15804 trim_cache size 0 max 16384 2023-12-12T10:01:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:01:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:01:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:01:03.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:01:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 -- 172.21.15.42:0/3735465260 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f04640a15f0 msgr2=0x7f04640a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f04640a15f0 0x7f04640a3ab0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0474005950 tx=0x7f047400b410 comp rx=0 tx=0).stop 2023-12-12T10:01:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 -- 172.21.15.42:0/3735465260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106080 msgr2=0x7f047819dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106080 0x7f047819dd30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f046800eb30 tx=0x7f046800ee40 comp rx=0 tx=0).stop 2023-12-12T10:01:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 -- 172.21.15.42:0/3735465260 shutdown_connections 2023-12-12T10:01:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0478106a40 0x7f0478197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0478068ca0 0x7f047819d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f04640a15f0 0x7f04640a3ab0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 --2- 172.21.15.42:0/3735465260 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0478106080 0x7f047819dd30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 -- 172.21.15.42:0/3735465260 >> 172.21.15.42:0/3735465260 conn(0x7f0478077570 msgr2=0x7f04780fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 -- 172.21.15.42:0/3735465260 shutdown_connections 2023-12-12T10:01:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:03.373+0000 7f0480181700 1 -- 172.21.15.42:0/3735465260 wait complete. 2023-12-12T10:01:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:04 smithi042 ceph-mon[98894]: pgmap v76: 129 pgs: 129 active+clean; 9.8 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 785 KiB/s wr, 530 op/s 2023-12-12T10:01:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:04 smithi111 ceph-mon[102565]: pgmap v76: 129 pgs: 129 active+clean; 9.8 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 785 KiB/s wr, 530 op/s 2023-12-12T10:01:05.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:04 smithi173 ceph-mon[103308]: pgmap v76: 129 pgs: 129 active+clean; 9.8 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 559 KiB/s rd, 785 KiB/s wr, 530 op/s 2023-12-12T10:01:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:06 smithi042 ceph-mon[98894]: pgmap v77: 129 pgs: 129 active+clean; 9.8 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 774 KiB/s wr, 519 op/s 2023-12-12T10:01:06.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:06 smithi111 ceph-mon[102565]: pgmap v77: 129 pgs: 129 active+clean; 9.8 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 774 KiB/s wr, 519 op/s 2023-12-12T10:01:07.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:06 smithi173 ceph-mon[103308]: pgmap v77: 129 pgs: 129 active+clean; 9.8 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 774 KiB/s wr, 519 op/s 2023-12-12T10:01:08.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:08 smithi111 ceph-mon[102565]: pgmap v78: 129 pgs: 129 active+clean; 9.8 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 774 KiB/s wr, 519 op/s 2023-12-12T10:01:08.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:08 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:08 smithi173 ceph-mon[103308]: pgmap v78: 129 pgs: 129 active+clean; 9.8 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 774 KiB/s wr, 519 op/s 2023-12-12T10:01:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:08 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:08 smithi042 ceph-mon[98894]: pgmap v78: 129 pgs: 129 active+clean; 9.8 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 548 KiB/s rd, 774 KiB/s wr, 519 op/s 2023-12-12T10:01:09.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:08 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:10 smithi111 ceph-mon[102565]: pgmap v79: 129 pgs: 129 active+clean; 19 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2023-12-12T10:01:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:10 smithi173 ceph-mon[103308]: pgmap v79: 129 pgs: 129 active+clean; 19 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2023-12-12T10:01:11.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:10 smithi042 ceph-mon[98894]: pgmap v79: 129 pgs: 129 active+clean; 19 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 573 op/s 2023-12-12T10:01:12.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:12 smithi111 ceph-mon[102565]: pgmap v80: 129 pgs: 129 active+clean; 19 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 567 op/s 2023-12-12T10:01:13.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:12 smithi173 ceph-mon[103308]: pgmap v80: 129 pgs: 129 active+clean; 19 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 567 op/s 2023-12-12T10:01:13.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:12 smithi042 ceph-mon[98894]: pgmap v80: 129 pgs: 129 active+clean; 19 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 567 op/s 2023-12-12T10:01:14.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:14 smithi111 ceph-mon[102565]: pgmap v81: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 579 op/s 2023-12-12T10:01:15.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:14 smithi173 ceph-mon[103308]: pgmap v81: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 579 op/s 2023-12-12T10:01:15.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:14 smithi042 ceph-mon[98894]: pgmap v81: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 579 op/s 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.505+0000 7f1af9c96700 1 -- 172.21.15.42:0/3908477796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af4100c60 msgr2=0x7f1af410d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.505+0000 7f1af9c96700 1 --2- 172.21.15.42:0/3908477796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af4100c60 0x7f1af410d890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009990 tx=0x7f1ae8009ca0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.505+0000 7f1af9c96700 1 -- 172.21.15.42:0/3908477796 shutdown_connections 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.505+0000 7f1af9c96700 1 --2- 172.21.15.42:0/3908477796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af410de60 0x7f1af4110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.505+0000 7f1af9c96700 1 --2- 172.21.15.42:0/3908477796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1af41002b0 0x7f1af4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.505+0000 7f1af9c96700 1 --2- 172.21.15.42:0/3908477796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af4100c60 0x7f1af410d890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.505+0000 7f1af9c96700 1 -- 172.21.15.42:0/3908477796 >> 172.21.15.42:0/3908477796 conn(0x7f1af40fbea0 msgr2=0x7f1af40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.505+0000 7f1af9c96700 1 -- 172.21.15.42:0/3908477796 shutdown_connections 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.505+0000 7f1af9c96700 1 -- 172.21.15.42:0/3908477796 wait complete. 2023-12-12T10:01:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af9c96700 1 Processor -- start 2023-12-12T10:01:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af9c96700 1 -- start start 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af9c96700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af41002b0 0x7f1af419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af9c96700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4100c60 0x7f1af419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af9c96700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1af410de60 0x7f1af4199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af9c96700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af419a0c0 con 0x7f1af410de60 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af9c96700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af419a230 con 0x7f1af41002b0 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af9c96700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af419a3f0 con 0x7f1af4100c60 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af37fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af41002b0 0x7f1af419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1af410de60 0x7f1af4199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1af410de60 0x7f1af4199a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36310/0 (socket says 172.21.15.42:36310) 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af3fff700 1 -- 172.21.15.42:0/4016256865 learned_addr learned my addr 172.21.15.42:0/4016256865 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af37fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af41002b0 0x7f1af419f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39368/0 (socket says 172.21.15.42:39368) 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.506+0000 7f1af2ffd700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4100c60 0x7f1af419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af3fff700 1 -- 172.21.15.42:0/4016256865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4100c60 msgr2=0x7f1af419fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af3fff700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4100c60 0x7f1af419fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af3fff700 1 -- 172.21.15.42:0/4016256865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af41002b0 msgr2=0x7f1af419f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af3fff700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af41002b0 0x7f1af419f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af3fff700 1 -- 172.21.15.42:0/4016256865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae4009520 con 0x7f1af410de60 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af37fe700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af41002b0 0x7f1af419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af3fff700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1af410de60 0x7f1af4199a50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1aec007ce0 tx=0x7f1aec00d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af0ff9700 1 -- 172.21.15.42:0/4016256865 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec004db0 con 0x7f1af410de60 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af0ff9700 1 -- 172.21.15.42:0/4016256865 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aec00f040 con 0x7f1af410de60 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af9c96700 1 -- 172.21.15.42:0/4016256865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae80095f0 con 0x7f1af410de60 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af0ff9700 1 -- 172.21.15.42:0/4016256865 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec005990 con 0x7f1af410de60 2023-12-12T10:01:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.507+0000 7f1af9c96700 1 -- 172.21.15.42:0/4016256865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af41a66e0 con 0x7f1af410de60 2023-12-12T10:01:16.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.510+0000 7f1af0ff9700 1 -- 172.21.15.42:0/4016256865 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f1aec015400 con 0x7f1af410de60 2023-12-12T10:01:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.510+0000 7f1af0ff9700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f1adc0a15f0 0x7f1adc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.510+0000 7f1af37fe700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f1adc0a15f0 0x7f1adc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.510+0000 7f1af0ff9700 1 -- 172.21.15.42:0/4016256865 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1aec0ccc80 con 0x7f1af410de60 2023-12-12T10:01:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1af37fe700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f1adc0a15f0 0x7f1adc0a3ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009fd0 tx=0x7f1ae4009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1b05e8fb80 1 Processor -- start 2023-12-12T10:01:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1b05e8fb80 1 -- start start 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 0x55840f7cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7c9ac0 0x55840f7c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f724af0 con 0x55840f7cf660 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f722410 con 0x55840f7c9ac0 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f71dd00 con 0x55840f7ced40 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1af2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1af2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36316/0 (socket says 172.21.15.42:36316) 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1af2ffd700 1 -- 172.21.15.42:0/1125802617 learned_addr learned my addr 172.21.15.42:0/1125802617 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.511+0000 7f1af3fff700 1 --2- 172.21.15.42:0/1125802617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7c9ac0 0x55840f7c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1af37fe700 1 --2- 172.21.15.42:0/1125802617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 0x55840f7cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1af2ffd700 1 -- 172.21.15.42:0/1125802617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 msgr2=0x55840f7cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1af2ffd700 1 --2- 172.21.15.42:0/1125802617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 0x55840f7cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1af2ffd700 1 -- 172.21.15.42:0/1125802617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7c9ac0 msgr2=0x55840f7c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1af2ffd700 1 --2- 172.21.15.42:0/1125802617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7c9ac0 0x55840f7c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1af2ffd700 1 -- 172.21.15.42:0/1125802617 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840f59a5d0 con 0x55840f7cf660 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1af2ffd700 1 --2- 172.21.15.42:0/1125802617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7c9580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009990 tx=0x7f1ae8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551f6b08aedbe984 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1ad9ffb700 1 -- 172.21.15.42:0/1125802617 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae801d070 con 0x55840f7cf660 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1ad9ffb700 1 -- 172.21.15.42:0/1125802617 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ae801e040 con 0x55840f7cf660 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.512+0000 7f1ad9ffb700 1 -- 172.21.15.42:0/1125802617 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8004880 con 0x55840f7cf660 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1125802617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 msgr2=0x55840f7c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/1125802617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7c9580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009990 tx=0x7f1ae8005bc0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1125802617 shutdown_connections 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/1125802617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7c9ac0 0x55840f7c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/1125802617 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 0x55840f7cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/1125802617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7c9580 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1125802617 >> 172.21.15.42:0/1125802617 conn(0x55840f6f8780 msgr2=0x55840f7c3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1125802617 shutdown_connections 2023-12-12T10:01:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1125802617 wait complete. 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.513+0000 7f1b05e8fb80 1 -- start start 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1b05e8fb80 1 Processor -- start 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 0x55840f7d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7cf660 0x55840f7d3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f6d7500 con 0x55840f7c9ac0 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f6d71a0 con 0x55840f7cf660 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f59c8c0 con 0x55840f7ced40 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1b05e8fb80 10 client.0 _ll_register_callbacks cb 0x55840f602a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7cf660 0x55840f7d3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7cf660 0x55840f7d3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39384/0 (socket says 172.21.15.42:39384) 2023-12-12T10:01:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af3fff700 1 -- 172.21.15.42:0/2938337414 learned_addr learned my addr 172.21.15.42:0/2938337414 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af2ffd700 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 0x55840f7d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af3fff700 1 -- 172.21.15.42:0/2938337414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 msgr2=0x55840f7d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af3fff700 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 0x55840f7d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af3fff700 1 -- 172.21.15.42:0/2938337414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 msgr2=0x55840f7d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af3fff700 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af3fff700 1 -- 172.21.15.42:0/2938337414 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af4110ec0 con 0x55840f7cf660 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1af3fff700 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7cf660 0x55840f7d3410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1aec095e10 tx=0x7f1aec0997b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a3d81e26f63294 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.514+0000 7f1ad2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2938337414 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0b92b0 con 0x55840f7cf660 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2938337414 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55840f7bc730 con 0x55840f7cf660 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2938337414 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1aec09a010 con 0x55840f7cf660 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2938337414 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aec099910 con 0x55840f7cf660 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1b05e8fb80 10 client.25499 fetch_fsmap learned FSMap version 28 2023-12-12T10:01:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2938337414 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55840f546d80 con 0x55840f7cf660 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2938337414 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55840f7bc730 con 0x55840f7cf660 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2938337414 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1aec098e00 con 0x55840f7cf660 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1b05e8fb80 10 client.25499 fetch_fsmap finished waiting for FSMap version 28 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1b05e8fb80 10 client.25499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1b05e8fb80 20 client.25499 populate_metadata read hostname 'smithi042' 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55840f7e3140 0x55840f7e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1b05e8fb80 4 client.25499 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2938337414 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55840f798ee0 con 0x55840f7e3140 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.515+0000 7f1af37fe700 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55840f7e3140 0x55840f7e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.516+0000 7f1af37fe700 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55840f7e3140 0x55840f7e5560 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1db688c1024ed3 server_cookie=dab98ae1abb73c2d in_seq=0 out_seq=0 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.516+0000 7f1ad2ffd700 10 client.25499 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2938337414 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55840f798ee0 con 0x55840f7e3140 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1ad2ffd700 10 client.25499 handle_command_reply: tid=0 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 client.25499 shutdown 2023-12-12T10:01:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2938337414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7cf660 msgr2=0x55840f7d3410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7cf660 0x55840f7d3410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1aec095e10 tx=0x7f1aec0997b0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2938337414 shutdown_connections 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55840f7e3140 0x55840f7e5560 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7cf660 0x55840f7d3410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1ad2ffd700 0 client.25499 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2938337414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7ced40 0x55840f7d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2938337414 >> 172.21.15.42:0/2938337414 conn(0x55840f6f8780 msgr2=0x55840f7c8570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2938337414 shutdown_connections 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2938337414 wait complete. 2023-12-12T10:01:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.517+0000 7f1b05e8fb80 20 client.25499 trim_cache size 0 max 16384 2023-12-12T10:01:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.532+0000 7f1b05e8fb80 1 Processor -- start 2023-12-12T10:01:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.532+0000 7f1b05e8fb80 1 -- start start 2023-12-12T10:01:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.532+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7c9ac0 0x55840f7d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.532+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 0x55840f7d9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.532+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7da520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.532+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f798ee0 con 0x55840f7cf660 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.532+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f7d5960 con 0x55840f7ced40 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.532+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f7d3f90 con 0x55840f7c9ac0 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7da520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7da520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36332/0 (socket says 172.21.15.42:36332) 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af3fff700 1 -- 172.21.15.42:0/1910312990 learned_addr learned my addr 172.21.15.42:0/1910312990 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af2ffd700 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 0x55840f7d9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af37fe700 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7c9ac0 0x55840f7d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af3fff700 1 -- 172.21.15.42:0/1910312990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7c9ac0 msgr2=0x55840f7d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af3fff700 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7c9ac0 0x55840f7d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af3fff700 1 -- 172.21.15.42:0/1910312990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 msgr2=0x55840f7d9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af3fff700 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 0x55840f7d9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af3fff700 1 -- 172.21.15.42:0/1910312990 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840f52fc00 con 0x55840f7cf660 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af37fe700 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7c9ac0 0x55840f7d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:01:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1af3fff700 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7da520 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1aec004b80 tx=0x7f1aec0053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a081f755e724f10b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1ada7fc700 1 -- 172.21.15.42:0/1910312990 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec099450 con 0x55840f7cf660 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1ada7fc700 1 -- 172.21.15.42:0/1910312990 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aec09a010 con 0x55840f7cf660 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.533+0000 7f1ada7fc700 1 -- 172.21.15.42:0/1910312990 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec098e00 con 0x55840f7cf660 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1910312990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 msgr2=0x55840f7da520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7da520 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1aec004b80 tx=0x7f1aec0053f0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1910312990 shutdown_connections 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7cf660 0x55840f7da520 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7c9ac0 0x55840f7d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/1910312990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 0x55840f7d9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1910312990 >> 172.21.15.42:0/1910312990 conn(0x55840f6f8780 msgr2=0x55840f7c3c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1910312990 shutdown_connections 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/1910312990 wait complete. 2023-12-12T10:01:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 -- start start 2023-12-12T10:01:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.534+0000 7f1b05e8fb80 1 Processor -- start 2023-12-12T10:01:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 0x55840f7d86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1b05e8fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7cf660 0x55840f838640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f838b80 con 0x55840f7c9ac0 2023-12-12T10:01:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f838cc0 con 0x55840f7ced40 2023-12-12T10:01:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1b05e8fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840f838e50 con 0x55840f7cf660 2023-12-12T10:01:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1b05e8fb80 10 client.0 _ll_register_callbacks cb 0x55840f602a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36334/0 (socket says 172.21.15.42:36334) 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af37fe700 1 -- 172.21.15.42:0/2175551333 learned_addr learned my addr 172.21.15.42:0/2175551333 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af2ffd700 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 0x55840f7d86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af3fff700 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7cf660 0x55840f838640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af37fe700 1 -- 172.21.15.42:0/2175551333 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7cf660 msgr2=0x55840f838640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af37fe700 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7cf660 0x55840f838640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af37fe700 1 -- 172.21.15.42:0/2175551333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 msgr2=0x55840f7d86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af37fe700 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 0x55840f7d86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af37fe700 1 -- 172.21.15.42:0/2175551333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55840f546d80 con 0x55840f7c9ac0 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af3fff700 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7cf660 0x55840f838640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1af37fe700 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d81b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4011d50 tx=0x7f1ae40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c1b90e17b46d9f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1ad2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:01:16.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2175551333 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4020040 con 0x55840f7c9ac0 2023-12-12T10:01:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2175551333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ae801e040 con 0x55840f7c9ac0 2023-12-12T10:01:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.535+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2175551333 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1ae400e910 con 0x55840f7c9ac0 2023-12-12T10:01:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2175551333 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ae400eb10 con 0x55840f7c9ac0 2023-12-12T10:01:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1b05e8fb80 10 client.15822 fetch_fsmap learned FSMap version 28 2023-12-12T10:01:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2175551333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55840f56d1f0 con 0x55840f7c9ac0 2023-12-12T10:01:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2175551333 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ae801e040 con 0x55840f7c9ac0 2023-12-12T10:01:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2175551333 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 28) v1 ==== 7894+0+0 (secure 0 0 0) 0x7f1ae4031020 con 0x55840f7c9ac0 2023-12-12T10:01:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1b05e8fb80 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2023-12-12T10:01:16.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1b05e8fb80 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:01:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1b05e8fb80 20 client.15822 populate_metadata read hostname 'smithi042' 2023-12-12T10:01:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55840f83a720 0x55840f83cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1b05e8fb80 4 client.15822 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:01:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2175551333 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55840f6e3f70 con 0x55840f83a720 2023-12-12T10:01:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.536+0000 7f1af2ffd700 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55840f83a720 0x55840f83cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.537+0000 7f1ad2ffd700 10 client.15822 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:16.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:16.537+0000 7f1af2ffd700 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55840f83a720 0x55840f83cb40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa7daca4b9714 server_cookie=a0fff69905304590 in_seq=0 out_seq=0 2023-12-12T10:01:16.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:16 smithi111 ceph-mon[102565]: pgmap v82: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 66 op/s 2023-12-12T10:01:17.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:16 smithi173 ceph-mon[103308]: pgmap v82: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 66 op/s 2023-12-12T10:01:17.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:16 smithi042 ceph-mon[98894]: pgmap v82: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 66 op/s 2023-12-12T10:01:18.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:18 smithi111 ceph-mon[102565]: pgmap v83: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 66 op/s 2023-12-12T10:01:18.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:18 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:19.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:18 smithi173 ceph-mon[103308]: pgmap v83: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 66 op/s 2023-12-12T10:01:19.082 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:18 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:19.161 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:18 smithi042 ceph-mon[98894]: pgmap v83: 129 pgs: 129 active+clean; 33 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 66 op/s 2023-12-12T10:01:19.161 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:18 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:21.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:20 smithi173 ceph-mon[103308]: pgmap v84: 129 pgs: 129 active+clean; 44 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 838 op/s 2023-12-12T10:01:21.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:20 smithi042 ceph-mon[98894]: pgmap v84: 129 pgs: 129 active+clean; 44 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 838 op/s 2023-12-12T10:01:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1ad2ffd700 1 -- 172.21.15.42:0/2175551333 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55840f6e3f70 con 0x55840f83a720 2023-12-12T10:01:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1ad2ffd700 10 client.15822 handle_command_reply: tid=0 2023-12-12T10:01:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 client.15822 shutdown 2023-12-12T10:01:21.210 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2175551333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 msgr2=0x55840f7d81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d81b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4011d50 tx=0x7f1ae40047c0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2175551333 shutdown_connections 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55840f7cf660 0x55840f838640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55840f7c9ac0 0x55840f7d81b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55840f83a720 0x55840f83cb40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 --2- 172.21.15.42:0/2175551333 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55840f7ced40 0x55840f7d86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1ad2ffd700 0 client.15822 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2175551333 >> 172.21.15.42:0/2175551333 conn(0x55840f6f8780 msgr2=0x55840f7d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2175551333 shutdown_connections 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.207+0000 7f1b05e8fb80 1 -- 172.21.15.42:0/2175551333 wait complete. 2023-12-12T10:01:21.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1b05e8fb80 20 client.15822 trim_cache size 0 max 16384 2023-12-12T10:01:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:01:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:01:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:01:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:01:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 -- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f1adc0a15f0 msgr2=0x7f1adc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f1adc0a15f0 0x7f1adc0a3ab0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009fd0 tx=0x7f1ae4009260 comp rx=0 tx=0).stop 2023-12-12T10:01:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 -- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1af410de60 msgr2=0x7f1af4199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1af410de60 0x7f1af4199a50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1aec007ce0 tx=0x7f1aec00d0d0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 -- 172.21.15.42:0/4016256865 shutdown_connections 2023-12-12T10:01:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f1adc0a15f0 0x7f1adc0a3ab0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1af41002b0 0x7f1af419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1af410de60 0x7f1af4199a50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 --2- 172.21.15.42:0/4016256865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1af4100c60 0x7f1af419fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 -- 172.21.15.42:0/4016256865 >> 172.21.15.42:0/4016256865 conn(0x7f1af40fbea0 msgr2=0x7f1af40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 -- 172.21.15.42:0/4016256865 shutdown_connections 2023-12-12T10:01:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:21.208+0000 7f1af9c96700 1 -- 172.21.15.42:0/4016256865 wait complete. 2023-12-12T10:01:21.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:20 smithi111 ceph-mon[102565]: pgmap v84: 129 pgs: 129 active+clean; 44 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.9 MiB/s wr, 838 op/s 2023-12-12T10:01:22.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:22 smithi111 ceph-mon[102565]: pgmap v85: 129 pgs: 129 active+clean; 44 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 783 op/s 2023-12-12T10:01:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:22 smithi173 ceph-mon[103308]: pgmap v85: 129 pgs: 129 active+clean; 44 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 783 op/s 2023-12-12T10:01:23.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:22 smithi042 ceph-mon[98894]: pgmap v85: 129 pgs: 129 active+clean; 44 MiB data, 494 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.5 MiB/s wr, 783 op/s 2023-12-12T10:01:24.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:23 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:24.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:23 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:24.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:23 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:25.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:24 smithi173 ceph-mon[103308]: pgmap v86: 129 pgs: 129 active+clean; 32 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 1.43k op/s 2023-12-12T10:01:25.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:24 smithi042 ceph-mon[98894]: pgmap v86: 129 pgs: 129 active+clean; 32 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 1.43k op/s 2023-12-12T10:01:25.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:24 smithi111 ceph-mon[102565]: pgmap v86: 129 pgs: 129 active+clean; 32 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 1.43k op/s 2023-12-12T10:01:27.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:26 smithi173 ceph-mon[103308]: pgmap v87: 129 pgs: 129 active+clean; 14 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 3.37k op/s 2023-12-12T10:01:27.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:26 smithi042 ceph-mon[98894]: pgmap v87: 129 pgs: 129 active+clean; 14 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 3.37k op/s 2023-12-12T10:01:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:26 smithi111 ceph-mon[102565]: pgmap v87: 129 pgs: 129 active+clean; 14 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 3.37k op/s 2023-12-12T10:01:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:28 smithi173 ceph-mon[103308]: pgmap v88: 129 pgs: 129 active+clean; 14 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 3.37k op/s 2023-12-12T10:01:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:28 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:29.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:28 smithi042 ceph-mon[98894]: pgmap v88: 129 pgs: 129 active+clean; 14 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 3.37k op/s 2023-12-12T10:01:29.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:28 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:29.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:28 smithi111 ceph-mon[102565]: pgmap v88: 129 pgs: 129 active+clean; 14 MiB data, 606 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 3.37k op/s 2023-12-12T10:01:29.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:28 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:01:31.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:30 smithi173 ceph-mon[103308]: pgmap v89: 129 pgs: 129 active+clean; 28 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 3.40k op/s 2023-12-12T10:01:31.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:30 smithi042 ceph-mon[98894]: pgmap v89: 129 pgs: 129 active+clean; 28 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 3.40k op/s 2023-12-12T10:01:31.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:30 smithi111 ceph-mon[102565]: pgmap v89: 129 pgs: 129 active+clean; 28 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 4.0 MiB/s wr, 3.40k op/s 2023-12-12T10:01:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:32 smithi173 ceph-mon[103308]: pgmap v90: 129 pgs: 129 active+clean; 28 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 2.63k op/s 2023-12-12T10:01:33.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:32 smithi042 ceph-mon[98894]: pgmap v90: 129 pgs: 129 active+clean; 28 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 2.63k op/s 2023-12-12T10:01:33.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:32 smithi111 ceph-mon[102565]: pgmap v90: 129 pgs: 129 active+clean; 28 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 2.63k op/s 2023-12-12T10:01:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.333+0000 7f4e81c49700 1 -- 172.21.15.42:0/4148935223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c1003d0 msgr2=0x7f4e7c100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.333+0000 7f4e81c49700 1 --2- 172.21.15.42:0/4148935223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c1003d0 0x7f4e7c100850 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009990 tx=0x7f4e70009ca0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.333+0000 7f4e81c49700 1 -- 172.21.15.42:0/4148935223 shutdown_connections 2023-12-12T10:01:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.333+0000 7f4e81c49700 1 --2- 172.21.15.42:0/4148935223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c106470 0x7f4e7c106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.333+0000 7f4e81c49700 1 --2- 172.21.15.42:0/4148935223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e7c100d90 0x7f4e7c10d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.333+0000 7f4e81c49700 1 --2- 172.21.15.42:0/4148935223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c1003d0 0x7f4e7c100850 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.333+0000 7f4e81c49700 1 -- 172.21.15.42:0/4148935223 >> 172.21.15.42:0/4148935223 conn(0x7f4e7c077570 msgr2=0x7f4e7c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:34.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.333+0000 7f4e81c49700 1 -- 172.21.15.42:0/4148935223 shutdown_connections 2023-12-12T10:01:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.333+0000 7f4e81c49700 1 -- 172.21.15.42:0/4148935223 wait complete. 2023-12-12T10:01:34.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.334+0000 7f4e81c49700 1 Processor -- start 2023-12-12T10:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.334+0000 7f4e81c49700 1 -- start start 2023-12-12T10:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.334+0000 7f4e81c49700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e7c1003d0 0x7f4e7c19f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.334+0000 7f4e81c49700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c100d90 0x7f4e7c19fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.334+0000 7f4e81c49700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c106470 0x7f4e7c199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.334+0000 7f4e81c49700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c19a120 con 0x7f4e7c106470 2023-12-12T10:01:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.334+0000 7f4e81c49700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c19a290 con 0x7f4e7c100d90 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.334+0000 7f4e81c49700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c19a450 con 0x7f4e7c1003d0 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e7c1003d0 0x7f4e7c19f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7affd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c100d90 0x7f4e7c19fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e7c1003d0 0x7f4e7c19f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51336/0 (socket says 172.21.15.42:51336) 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/1876843084 learned_addr learned my addr 172.21.15.42:0/1876843084 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7bfff700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c106470 0x7f4e7c199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/1876843084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c100d90 msgr2=0x7f4e7c19fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c100d90 0x7f4e7c19fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c106470 msgr2=0x7f4e7c199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c106470 0x7f4e7c199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/1876843084 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e700095f0 con 0x7f4e7c1003d0 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e7c1003d0 0x7f4e7c19f860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00e9a0 tx=0x7f4e6c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e78ff9700 1 -- 172.21.15.42:0/1876843084 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c009790 con 0x7f4e7c1003d0 2023-12-12T10:01:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e81c49700 1 -- 172.21.15.42:0/1876843084 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7c19a730 con 0x7f4e7c1003d0 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e78ff9700 1 -- 172.21.15.42:0/1876843084 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e6c00cbd0 con 0x7f4e7c1003d0 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e78ff9700 1 -- 172.21.15.42:0/1876843084 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c018680 con 0x7f4e7c1003d0 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.335+0000 7f4e81c49700 1 -- 172.21.15.42:0/1876843084 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7c1a67e0 con 0x7f4e7c1003d0 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.338+0000 7f4e78ff9700 1 -- 172.21.15.42:0/1876843084 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f4e6c00cd40 con 0x7f4e7c1003d0 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.339+0000 7f4e78ff9700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e640a15f0 0x7f4e640a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.339+0000 7f4e78ff9700 1 -- 172.21.15.42:0/1876843084 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4e6c014070 con 0x7f4e7c1003d0 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.339+0000 7f4e7affd700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e640a15f0 0x7f4e640a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.339+0000 7f4e8de42b80 1 Processor -- start 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.339+0000 7f4e8de42b80 1 -- start start 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.339+0000 7f4e7affd700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e640a15f0 0x7f4e640a3ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c19b070 tx=0x7f4e70005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc543a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 0x556bc5434580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5434f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc538faf0 con 0x556bc5434ac0 2023-12-12T10:01:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc538d410 con 0x556bc5439d40 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc5388d00 con 0x556bc543a660 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7affd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 0x556bc5434580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc543a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc543a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39420/0 (socket says 172.21.15.42:39420) 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7affd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 0x556bc5434580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51346/0 (socket says 172.21.15.42:51346) 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/3912163604 learned_addr learned my addr 172.21.15.42:0/3912163604 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7bfff700 1 --2- 172.21.15.42:0/3912163604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5434f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/3912163604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 msgr2=0x556bc5434580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/3912163604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 0x556bc5434580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/3912163604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 msgr2=0x556bc5434f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/3912163604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5434f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.340+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/3912163604 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bc52055d0 con 0x556bc5439d40 2023-12-12T10:01:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.341+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/3912163604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc543a120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009230 tx=0x7f4e6c099c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fedbb3df9891764f server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.341+0000 7f4e61ffb700 1 -- 172.21.15.42:0/3912163604 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c09a300 con 0x556bc5439d40 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.341+0000 7f4e61ffb700 1 -- 172.21.15.42:0/3912163604 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e6c0b0070 con 0x556bc5439d40 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.341+0000 7f4e61ffb700 1 -- 172.21.15.42:0/3912163604 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c09a560 con 0x556bc5439d40 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 -- 172.21.15.42:0/3912163604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 msgr2=0x556bc543a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/3912163604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc543a120 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c009230 tx=0x7f4e6c099c60 comp rx=0 tx=0).stop 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 -- 172.21.15.42:0/3912163604 shutdown_connections 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/3912163604 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc543a120 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/3912163604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5434f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/3912163604 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 0x556bc5434580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 -- 172.21.15.42:0/3912163604 >> 172.21.15.42:0/3912163604 conn(0x556bc5363780 msgr2=0x556bc542ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 -- 172.21.15.42:0/3912163604 shutdown_connections 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 -- 172.21.15.42:0/3912163604 wait complete. 2023-12-12T10:01:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 -- start start 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.342+0000 7f4e8de42b80 1 Processor -- start 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc54405d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc543d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 0x556bc543df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc5342500 con 0x556bc5434ac0 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc53421a0 con 0x556bc543a660 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc52078c0 con 0x556bc5439d40 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e8de42b80 10 client.0 _ll_register_callbacks cb 0x556bc526da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc54405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc54405d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36366/0 (socket says 172.21.15.42:36366) 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/361730386 learned_addr learned my addr 172.21.15.42:0/361730386 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7affd700 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc543d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7bfff700 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 0x556bc543df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/361730386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 msgr2=0x556bc543d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc543d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/361730386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 msgr2=0x556bc543df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 0x556bc543df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/361730386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7c1a67e0 con 0x556bc5434ac0 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc54405d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0cab20 tx=0x7f4e6c09e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc04af89bf3129f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e5affd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.343+0000 7f4e5affd700 1 -- 172.21.15.42:0/361730386 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c09a300 con 0x556bc5434ac0 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.344+0000 7f4e5affd700 1 -- 172.21.15.42:0/361730386 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4e6c0997f0 con 0x556bc5434ac0 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.344+0000 7f4e8de42b80 1 -- 172.21.15.42:0/361730386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bc5427730 con 0x556bc5434ac0 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.344+0000 7f4e5affd700 1 -- 172.21.15.42:0/361730386 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e6c0999b0 con 0x556bc5434ac0 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.344+0000 7f4e8de42b80 10 client.15834 fetch_fsmap learned FSMap version 31 2023-12-12T10:01:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.344+0000 7f4e8de42b80 1 -- 172.21.15.42:0/361730386 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556bc51b1d80 con 0x556bc5434ac0 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e5affd700 1 -- 172.21.15.42:0/361730386 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556bc5427730 con 0x556bc5434ac0 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e5affd700 1 -- 172.21.15.42:0/361730386 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4e6c099260 con 0x556bc5434ac0 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e8de42b80 10 client.15834 fetch_fsmap finished waiting for FSMap version 31 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e8de42b80 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e8de42b80 20 client.15834 populate_metadata read hostname 'smithi042' 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x556bc544dd10 0x556bc5450130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e8de42b80 4 client.15834 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e8de42b80 1 -- 172.21.15.42:0/361730386 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bc5403ee0 con 0x556bc544dd10 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e7affd700 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x556bc544dd10 0x556bc5450130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e5affd700 10 client.15834 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.345+0000 7f4e7affd700 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x556bc544dd10 0x556bc5450130 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1de457ecfd0011d2 server_cookie=968b87626a9eeff0 in_seq=0 out_seq=0 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e5affd700 1 -- 172.21.15.42:0/361730386 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556bc5403ee0 con 0x556bc544dd10 2023-12-12T10:01:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e5affd700 10 client.15834 handle_command_reply: tid=0 2023-12-12T10:01:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 client.15834 shutdown 2023-12-12T10:01:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 -- 172.21.15.42:0/361730386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 msgr2=0x556bc54405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc54405d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c0cab20 tx=0x7f4e6c09e450 comp rx=0 tx=0).stop 2023-12-12T10:01:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 -- 172.21.15.42:0/361730386 shutdown_connections 2023-12-12T10:01:34.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x556bc544dd10 0x556bc5450130 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 0x556bc543df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc54405d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/361730386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc543d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 -- 172.21.15.42:0/361730386 >> 172.21.15.42:0/361730386 conn(0x556bc5363780 msgr2=0x556bc5433570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e5affd700 0 client.15834 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 -- 172.21.15.42:0/361730386 shutdown_connections 2023-12-12T10:01:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.347+0000 7f4e8de42b80 1 -- 172.21.15.42:0/361730386 wait complete. 2023-12-12T10:01:34.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.348+0000 7f4e8de42b80 20 client.15834 trim_cache size 0 max 16384 2023-12-12T10:01:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e8de42b80 1 Processor -- start 2023-12-12T10:01:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e8de42b80 1 -- start start 2023-12-12T10:01:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5433d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc5454a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 0x556bc5445040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc5434930 con 0x556bc5434ac0 2023-12-12T10:01:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc544daf0 con 0x556bc543a660 2023-12-12T10:01:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc543e520 con 0x556bc5439d40 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5433d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7affd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc5454a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5433d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36374/0 (socket says 172.21.15.42:36374) 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/226899727 learned_addr learned my addr 172.21.15.42:0/226899727 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7affd700 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc5454a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51360/0 (socket says 172.21.15.42:51360) 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7bfff700 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 0x556bc5445040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7affd700 1 -- 172.21.15.42:0/226899727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 msgr2=0x556bc5445040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7affd700 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 0x556bc5445040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7affd700 1 -- 172.21.15.42:0/226899727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 msgr2=0x556bc5433d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7affd700 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5433d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7affd700 1 -- 172.21.15.42:0/226899727 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bc519ac00 con 0x556bc5439d40 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.363+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5433d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:01:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.364+0000 7f4e7affd700 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc5454a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000b2e0 tx=0x7f4e70020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f464ff0bd3b8aa4b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.364+0000 7f4e627fc700 1 -- 172.21.15.42:0/226899727 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7000b830 con 0x556bc5439d40 2023-12-12T10:01:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.364+0000 7f4e627fc700 1 -- 172.21.15.42:0/226899727 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e7001e040 con 0x556bc5439d40 2023-12-12T10:01:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.364+0000 7f4e627fc700 1 -- 172.21.15.42:0/226899727 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70004a40 con 0x556bc5439d40 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 -- 172.21.15.42:0/226899727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 msgr2=0x556bc5454a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc5454a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e7000b2e0 tx=0x7f4e70020000 comp rx=0 tx=0).stop 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 -- 172.21.15.42:0/226899727 shutdown_connections 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc543a660 0x556bc5445040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5433d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/226899727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc5439d40 0x556bc5454a70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 -- 172.21.15.42:0/226899727 >> 172.21.15.42:0/226899727 conn(0x556bc5363780 msgr2=0x556bc5440300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 -- 172.21.15.42:0/226899727 shutdown_connections 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 -- 172.21.15.42:0/226899727 wait complete. 2023-12-12T10:01:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 -- start start 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.365+0000 7f4e8de42b80 1 Processor -- start 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5445540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc5443020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e8de42b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 0x556bc5443740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc5443e50 con 0x556bc5434ac0 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc54a31f0 con 0x556bc5439d40 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e8de42b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bc54a3330 con 0x556bc543a660 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e8de42b80 10 client.0 _ll_register_callbacks cb 0x556bc526da20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5445540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7affd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc5443020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 0x556bc5443740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7b7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5445540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36380/0 (socket says 172.21.15.42:36380) 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7affd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc5443020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39442/0 (socket says 172.21.15.42:39442) 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/1653018796 learned_addr learned my addr 172.21.15.42:0/1653018796 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/1653018796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 msgr2=0x556bc5443740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 0x556bc5443740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/1653018796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 msgr2=0x556bc5443020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc5443020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7b7fe700 1 -- 172.21.15.42:0/1653018796 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bc51b1d80 con 0x556bc5434ac0 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e7b7fe700 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5445540 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c5b0 tx=0x7f4e6c09aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375388763cfad1ce server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e5affd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e5affd700 1 -- 172.21.15.42:0/1653018796 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c09a760 con 0x556bc5434ac0 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e5affd700 1 -- 172.21.15.42:0/1653018796 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4e6c09a300 con 0x556bc5434ac0 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.366+0000 7f4e8de42b80 1 -- 172.21.15.42:0/1653018796 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e6c0b0070 con 0x556bc5434ac0 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e5affd700 1 -- 172.21.15.42:0/1653018796 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e6c0a2690 con 0x556bc5434ac0 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e8de42b80 10 client.15840 fetch_fsmap learned FSMap version 31 2023-12-12T10:01:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e8de42b80 1 -- 172.21.15.42:0/1653018796 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f4e7c10dba0 con 0x556bc5434ac0 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e5affd700 1 -- 172.21.15.42:0/1653018796 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e6c0a8020 con 0x556bc5434ac0 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e5affd700 1 -- 172.21.15.42:0/1653018796 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7f4e6c09f220 con 0x556bc5434ac0 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e8de42b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 31 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e8de42b80 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e8de42b80 20 client.15840 populate_metadata read hostname 'smithi042' 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x556bc54a4770 0x556bc54a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e8de42b80 4 client.15840 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e8de42b80 1 -- 172.21.15.42:0/1653018796 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556bc534ef70 con 0x556bc54a4770 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.367+0000 7f4e7affd700 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x556bc54a4770 0x556bc54a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.368+0000 7f4e7affd700 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x556bc54a4770 0x556bc54a6b90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a085aedacf21d18 server_cookie=435f44f6b9d55e75 in_seq=0 out_seq=0 2023-12-12T10:01:34.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:34.368+0000 7f4e5affd700 10 client.15840 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:35.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:34 smithi173 ceph-mon[103308]: pgmap v91: 129 pgs: 129 active+clean; 36 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 2.68k op/s 2023-12-12T10:01:35.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:34 smithi042 ceph-mon[98894]: pgmap v91: 129 pgs: 129 active+clean; 36 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 2.68k op/s 2023-12-12T10:01:35.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:34 smithi111 ceph-mon[102565]: pgmap v91: 129 pgs: 129 active+clean; 36 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 2.68k op/s 2023-12-12T10:01:37.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:36 smithi173 ceph-mon[103308]: pgmap v92: 129 pgs: 129 active+clean; 60 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 2.15k op/s 2023-12-12T10:01:37.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:36 smithi042 ceph-mon[98894]: pgmap v92: 129 pgs: 129 active+clean; 60 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 2.15k op/s 2023-12-12T10:01:37.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:36 smithi111 ceph-mon[102565]: pgmap v92: 129 pgs: 129 active+clean; 60 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 5.2 MiB/s wr, 2.15k op/s 2023-12-12T10:01:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e5affd700 1 -- 172.21.15.42:0/1653018796 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556bc534ef70 con 0x556bc54a4770 2023-12-12T10:01:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e5affd700 10 client.15840 handle_command_reply: tid=0 2023-12-12T10:01:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 client.15840 shutdown 2023-12-12T10:01:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 -- 172.21.15.42:0/1653018796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 msgr2=0x556bc5445540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5445540 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00c5b0 tx=0x7f4e6c09aa00 comp rx=0 tx=0).stop 2023-12-12T10:01:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 -- 172.21.15.42:0/1653018796 shutdown_connections 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x556bc543a660 0x556bc5443740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556bc5434ac0 0x556bc5445540 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x556bc54a4770 0x556bc54a6b90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 --2- 172.21.15.42:0/1653018796 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x556bc5439d40 0x556bc5443020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e5affd700 0 client.15840 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 -- 172.21.15.42:0/1653018796 >> 172.21.15.42:0/1653018796 conn(0x556bc5363780 msgr2=0x556bc5449af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.410+0000 7f4e8de42b80 1 -- 172.21.15.42:0/1653018796 shutdown_connections 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e8de42b80 1 -- 172.21.15.42:0/1653018796 wait complete. 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e8de42b80 20 client.15840 trim_cache size 0 max 16384 2023-12-12T10:01:38.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:01:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:01:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:01:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:01:38.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 -- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e640a15f0 msgr2=0x7f4e640a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e640a15f0 0x7f4e640a3ab0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c19b070 tx=0x7f4e70005fd0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 -- 172.21.15.42:0/1876843084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e7c1003d0 msgr2=0x7f4e7c19f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e7c1003d0 0x7f4e7c19f860 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c00e9a0 tx=0x7f4e6c00ecb0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 -- 172.21.15.42:0/1876843084 shutdown_connections 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e7c106470 0x7f4e7c199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e7c1003d0 0x7f4e7c19f860 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e640a15f0 0x7f4e640a3ab0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 --2- 172.21.15.42:0/1876843084 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e7c100d90 0x7f4e7c19fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.411+0000 7f4e81c49700 1 -- 172.21.15.42:0/1876843084 >> 172.21.15.42:0/1876843084 conn(0x7f4e7c077570 msgr2=0x7f4e7c069600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.412+0000 7f4e81c49700 1 -- 172.21.15.42:0/1876843084 shutdown_connections 2023-12-12T10:01:38.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:38.412+0000 7f4e81c49700 1 -- 172.21.15.42:0/1876843084 wait complete. 2023-12-12T10:01:39.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:38 smithi173 ceph-mon[103308]: pgmap v93: 129 pgs: 129 active+clean; 60 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 200 op/s 2023-12-12T10:01:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:38 smithi042 ceph-mon[98894]: pgmap v93: 129 pgs: 129 active+clean; 60 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 200 op/s 2023-12-12T10:01:39.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:38 smithi111 ceph-mon[102565]: pgmap v93: 129 pgs: 129 active+clean; 60 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 200 op/s 2023-12-12T10:01:41.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:40 smithi173 ceph-mon[103308]: pgmap v94: 129 pgs: 129 active+clean; 71 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.9 MiB/s wr, 1.78k op/s 2023-12-12T10:01:41.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:40 smithi042 ceph-mon[98894]: pgmap v94: 129 pgs: 129 active+clean; 71 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.9 MiB/s wr, 1.78k op/s 2023-12-12T10:01:41.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:40 smithi111 ceph-mon[102565]: pgmap v94: 129 pgs: 129 active+clean; 71 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 5.9 MiB/s wr, 1.78k op/s 2023-12-12T10:01:43.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:42 smithi173 ceph-mon[103308]: pgmap v95: 129 pgs: 129 active+clean; 71 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 1.75k op/s 2023-12-12T10:01:43.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:42 smithi042 ceph-mon[98894]: pgmap v95: 129 pgs: 129 active+clean; 71 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 1.75k op/s 2023-12-12T10:01:43.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:42 smithi111 ceph-mon[102565]: pgmap v95: 129 pgs: 129 active+clean; 71 MiB data, 883 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.5 MiB/s wr, 1.75k op/s 2023-12-12T10:01:45.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:44 smithi173 ceph-mon[103308]: pgmap v96: 129 pgs: 129 active+clean; 69 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 1.76k op/s 2023-12-12T10:01:45.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:44 smithi042 ceph-mon[98894]: pgmap v96: 129 pgs: 129 active+clean; 69 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 1.76k op/s 2023-12-12T10:01:45.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:44 smithi111 ceph-mon[102565]: pgmap v96: 129 pgs: 129 active+clean; 69 MiB data, 888 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 1.76k op/s 2023-12-12T10:01:47.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:46 smithi173 ceph-mon[103308]: pgmap v97: 129 pgs: 129 active+clean; 55 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 1.76k op/s 2023-12-12T10:01:47.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:46 smithi042 ceph-mon[98894]: pgmap v97: 129 pgs: 129 active+clean; 55 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 1.76k op/s 2023-12-12T10:01:47.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:46 smithi111 ceph-mon[102565]: pgmap v97: 129 pgs: 129 active+clean; 55 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.6 MiB/s wr, 1.76k op/s 2023-12-12T10:01:49.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:48 smithi173 ceph-mon[103308]: pgmap v98: 129 pgs: 129 active+clean; 55 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 1.64k op/s 2023-12-12T10:01:49.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:48 smithi042 ceph-mon[98894]: pgmap v98: 129 pgs: 129 active+clean; 55 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 1.64k op/s 2023-12-12T10:01:49.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:48 smithi111 ceph-mon[102565]: pgmap v98: 129 pgs: 129 active+clean; 55 MiB data, 838 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.4 MiB/s wr, 1.64k op/s 2023-12-12T10:01:51.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:50 smithi173 ceph-mon[103308]: pgmap v99: 129 pgs: 129 active+clean; 72 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 1.79k op/s 2023-12-12T10:01:51.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:50 smithi042 ceph-mon[98894]: pgmap v99: 129 pgs: 129 active+clean; 72 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 1.79k op/s 2023-12-12T10:01:51.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:50 smithi111 ceph-mon[102565]: pgmap v99: 129 pgs: 129 active+clean; 72 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.3 MiB/s wr, 1.79k op/s 2023-12-12T10:01:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.531+0000 7fa50883f700 1 -- 172.21.15.42:0/2920558226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa504100f00 msgr2=0x7fa504104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.531+0000 7fa50883f700 1 --2- 172.21.15.42:0/2920558226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa504100f00 0x7fa504104f40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8009990 tx=0x7fa4f8009ca0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.531+0000 7fa50883f700 1 -- 172.21.15.42:0/2920558226 shutdown_connections 2023-12-12T10:01:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.531+0000 7fa50883f700 1 --2- 172.21.15.42:0/2920558226 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa504100550 0x7fa504100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.531+0000 7fa50883f700 1 --2- 172.21.15.42:0/2920558226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa504105670 0x7fa50410d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.531+0000 7fa50883f700 1 --2- 172.21.15.42:0/2920558226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa504100f00 0x7fa504104f40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.531+0000 7fa50883f700 1 -- 172.21.15.42:0/2920558226 >> 172.21.15.42:0/2920558226 conn(0x7fa5040fbdf0 msgr2=0x7fa5040fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.532+0000 7fa50883f700 1 -- 172.21.15.42:0/2920558226 shutdown_connections 2023-12-12T10:01:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.532+0000 7fa50883f700 1 -- 172.21.15.42:0/2920558226 wait complete. 2023-12-12T10:01:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.532+0000 7fa50883f700 1 Processor -- start 2023-12-12T10:01:51.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.532+0000 7fa50883f700 1 -- start start 2023-12-12T10:01:51.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa50883f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa504100550 0x7fa504196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa50883f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa504105670 0x7fa504196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa50883f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa504196fa0 0x7fa5041a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa50883f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041a60e0 con 0x7fa504196fa0 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa50883f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041a6220 con 0x7fa504105670 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa50883f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5041a6390 con 0x7fa504100550 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa501d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa504105670 0x7fa504196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa50259c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa504100550 0x7fa504196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa502d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa504196fa0 0x7fa5041a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa501d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa504105670 0x7fa504196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39458/0 (socket says 172.21.15.42:39458) 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa50259c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa504100550 0x7fa504196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51380/0 (socket says 172.21.15.42:51380) 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa501d9b700 1 -- 172.21.15.42:0/3514006574 learned_addr learned my addr 172.21.15.42:0/3514006574 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa501d9b700 1 -- 172.21.15.42:0/3514006574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa504100550 msgr2=0x7fa504196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:51.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa501d9b700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa504100550 0x7fa504196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa501d9b700 1 -- 172.21.15.42:0/3514006574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa504196fa0 msgr2=0x7fa5041a5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa501d9b700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa504196fa0 0x7fa5041a5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa501d9b700 1 -- 172.21.15.42:0/3514006574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f80095f0 con 0x7fa504105670 2023-12-12T10:01:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.533+0000 7fa501d9b700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa504105670 0x7fa504196a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8004760 tx=0x7fa4f8004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.534+0000 7fa4f37fe700 1 -- 172.21.15.42:0/3514006574 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f801d070 con 0x7fa504105670 2023-12-12T10:01:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.534+0000 7fa50883f700 1 -- 172.21.15.42:0/3514006574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5041a6610 con 0x7fa504105670 2023-12-12T10:01:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.534+0000 7fa4f37fe700 1 -- 172.21.15.42:0/3514006574 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4f8022470 con 0x7fa504105670 2023-12-12T10:01:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.534+0000 7fa4f37fe700 1 -- 172.21.15.42:0/3514006574 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8017650 con 0x7fa504105670 2023-12-12T10:01:51.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.534+0000 7fa50883f700 1 -- 172.21.15.42:0/3514006574 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5041a6a30 con 0x7fa504105670 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.537+0000 7fa514a38b80 1 Processor -- start 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.537+0000 7fa514a38b80 1 -- start start 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a418d40 0x55f76a419120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a419660 0x55f76a413580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a413f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa514a38b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a36eaf0 con 0x55f76a418d40 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa514a38b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a36c410 con 0x55f76a419660 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa514a38b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a367d00 con 0x55f76a413ac0 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa502d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a418d40 0x55f76a419120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa502d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a418d40 0x55f76a419120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36402/0 (socket says 172.21.15.42:36402) 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa502d9d700 1 -- 172.21.15.42:0/344198948 learned_addr learned my addr 172.21.15.42:0/344198948 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:51.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa50259c700 1 --2- 172.21.15.42:0/344198948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a419660 0x55f76a413580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa502d9d700 1 -- 172.21.15.42:0/344198948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 msgr2=0x55f76a413f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa502d9d700 1 --2- 172.21.15.42:0/344198948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a413f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa502d9d700 1 -- 172.21.15.42:0/344198948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a419660 msgr2=0x55f76a413580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa502d9d700 1 --2- 172.21.15.42:0/344198948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a419660 0x55f76a413580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa502d9d700 1 -- 172.21.15.42:0/344198948 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f76a1e45d0 con 0x55f76a418d40 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa50259c700 1 --2- 172.21.15.42:0/344198948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a419660 0x55f76a413580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa502d9d700 1 --2- 172.21.15.42:0/344198948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a418d40 0x55f76a419120 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00a730 tx=0x7fa4fc0078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc35189232efb58 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.538+0000 7fa4f0ff9700 1 -- 172.21.15.42:0/344198948 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc018070 con 0x55f76a418d40 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa4f0ff9700 1 -- 172.21.15.42:0/344198948 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4fc010040 con 0x55f76a418d40 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa4f0ff9700 1 -- 172.21.15.42:0/344198948 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00f050 con 0x55f76a418d40 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa4f37fe700 1 -- 172.21.15.42:0/3514006574 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fa4f80225e0 con 0x7fa504105670 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa4f37fe700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa4ec0a15e0 0x7fa4ec0a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa514a38b80 1 -- 172.21.15.42:0/344198948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a418d40 msgr2=0x55f76a419120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa514a38b80 1 --2- 172.21.15.42:0/344198948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a418d40 0x55f76a419120 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00a730 tx=0x7fa4fc0078d0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa514a38b80 1 -- 172.21.15.42:0/344198948 shutdown_connections 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa514a38b80 1 --2- 172.21.15.42:0/344198948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a413f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa514a38b80 1 --2- 172.21.15.42:0/344198948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a418d40 0x55f76a419120 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa514a38b80 1 --2- 172.21.15.42:0/344198948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a419660 0x55f76a413580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa514a38b80 1 -- 172.21.15.42:0/344198948 >> 172.21.15.42:0/344198948 conn(0x55f76a342780 msgr2=0x55f76a40dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.539+0000 7fa4f37fe700 1 -- 172.21.15.42:0/3514006574 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa4f80ce5a0 con 0x7fa504105670 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa50259c700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa4ec0a15e0 0x7fa4ec0a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 -- 172.21.15.42:0/344198948 shutdown_connections 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 -- 172.21.15.42:0/344198948 wait complete. 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 -- start start 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa50259c700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa4ec0a15e0 0x7fa4ec0a3aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55f76a40e8a0 tx=0x7fa4f4009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:51.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 Processor -- start 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a41ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a41cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a41d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a321500 con 0x55f76a419660 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a3211a0 con 0x55f76a418d40 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a1e68c0 con 0x55f76a413ac0 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.540+0000 7fa514a38b80 10 client.0 _ll_register_callbacks cb 0x55f76a24ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa502d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a41d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa501d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a41cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa502d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a41d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36412/0 (socket says 172.21.15.42:36412) 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa501d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a41cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39470/0 (socket says 172.21.15.42:39470) 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa502d9d700 1 -- 172.21.15.42:0/3975485625 learned_addr learned my addr 172.21.15.42:0/3975485625 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:51.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa50259c700 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a41ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa502d9d700 1 -- 172.21.15.42:0/3975485625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 msgr2=0x55f76a41ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa502d9d700 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a41ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa502d9d700 1 -- 172.21.15.42:0/3975485625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 msgr2=0x55f76a41cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa502d9d700 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a41cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa502d9d700 1 -- 172.21.15.42:0/3975485625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f40098e0 con 0x55f76a419660 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa501d9b700 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a41cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa502d9d700 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a41d500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc004430 tx=0x7fa4fc003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b333a5273c083 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa4e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/3975485625 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc018070 con 0x55f76a419660 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/3975485625 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa4fc004ad0 con 0x55f76a419660 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.541+0000 7fa514a38b80 1 -- 172.21.15.42:0/3975485625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f76a406730 con 0x55f76a419660 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/3975485625 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4fc006000 con 0x55f76a419660 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa514a38b80 10 client.15852 fetch_fsmap learned FSMap version 31 2023-12-12T10:01:51.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa514a38b80 1 -- 172.21.15.42:0/3975485625 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fa4fc00a410 con 0x55f76a419660 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/3975485625 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f76a406730 con 0x55f76a419660 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/3975485625 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa4fc01ed10 con 0x55f76a419660 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa514a38b80 10 client.15852 fetch_fsmap finished waiting for FSMap version 31 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa514a38b80 10 client.15852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa514a38b80 20 client.15852 populate_metadata read hostname 'smithi042' 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa514a38b80 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f76a42d0d0 0x55f76a42f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa514a38b80 4 client.15852 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa514a38b80 1 -- 172.21.15.42:0/3975485625 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f76a3e2ee0 con 0x55f76a42d0d0 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.542+0000 7fa50259c700 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f76a42d0d0 0x55f76a42f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.543+0000 7fa4e9ffb700 10 client.15852 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.543+0000 7fa50259c700 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f76a42d0d0 0x55f76a42f4f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3724f6a3db0e77c server_cookie=9fa9ecca9b4cc300 in_seq=0 out_seq=0 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.543+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/3975485625 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f76a3e2ee0 con 0x55f76a42d0d0 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.543+0000 7fa4e9ffb700 10 client.15852 handle_command_reply: tid=0 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.543+0000 7fa514a38b80 1 client.15852 shutdown 2023-12-12T10:01:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 -- 172.21.15.42:0/3975485625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 msgr2=0x55f76a41d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a41d500 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc004430 tx=0x7fa4fc003c90 comp rx=0 tx=0).stop 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 -- 172.21.15.42:0/3975485625 shutdown_connections 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a41d500 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a41ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f76a42d0d0 0x55f76a42f4f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 --2- 172.21.15.42:0/3975485625 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a41cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa4e9ffb700 0 client.15852 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 -- 172.21.15.42:0/3975485625 >> 172.21.15.42:0/3975485625 conn(0x55f76a342780 msgr2=0x55f76a412570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 -- 172.21.15.42:0/3975485625 shutdown_connections 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 1 -- 172.21.15.42:0/3975485625 wait complete. 2023-12-12T10:01:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.544+0000 7fa514a38b80 20 client.15852 trim_cache size 0 max 16384 2023-12-12T10:01:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa514a38b80 1 Processor -- start 2023-12-12T10:01:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa514a38b80 1 -- start start 2023-12-12T10:01:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a429140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a418440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a408950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa514a38b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a40d9f0 con 0x55f76a419660 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa514a38b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a40e8d0 con 0x55f76a418d40 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa514a38b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a41dfb0 con 0x55f76a413ac0 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa502d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a408950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa502d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a408950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36420/0 (socket says 172.21.15.42:36420) 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa501d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a418440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa502d9d700 1 -- 172.21.15.42:0/2728615891 learned_addr learned my addr 172.21.15.42:0/2728615891 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa50259c700 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a429140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa502d9d700 1 -- 172.21.15.42:0/2728615891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 msgr2=0x55f76a429140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa502d9d700 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a429140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa502d9d700 1 -- 172.21.15.42:0/2728615891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 msgr2=0x55f76a418440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa502d9d700 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a418440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa502d9d700 1 -- 172.21.15.42:0/2728615891 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f76a190d80 con 0x55f76a419660 2023-12-12T10:01:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa50259c700 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a429140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:01:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.559+0000 7fa501d9b700 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a418440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:01:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.560+0000 7fa502d9d700 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a408950 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc014450 tx=0x7fa4fc0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3165e7ad6ffb4b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.560+0000 7fa4f17fa700 1 -- 172.21.15.42:0/2728615891 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00f070 con 0x55f76a419660 2023-12-12T10:01:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.560+0000 7fa4f17fa700 1 -- 172.21.15.42:0/2728615891 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4fc019040 con 0x55f76a419660 2023-12-12T10:01:51.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.560+0000 7fa4f17fa700 1 -- 172.21.15.42:0/2728615891 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc018040 con 0x55f76a419660 2023-12-12T10:01:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 -- 172.21.15.42:0/2728615891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 msgr2=0x55f76a408950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a408950 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc014450 tx=0x7fa4fc0021d0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 -- 172.21.15.42:0/2728615891 shutdown_connections 2023-12-12T10:01:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a408950 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a413ac0 0x55f76a429140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2728615891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a418d40 0x55f76a418440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 -- 172.21.15.42:0/2728615891 >> 172.21.15.42:0/2728615891 conn(0x55f76a342780 msgr2=0x55f76a408240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 -- 172.21.15.42:0/2728615891 shutdown_connections 2023-12-12T10:01:51.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 -- 172.21.15.42:0/2728615891 wait complete. 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 -- start start 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 Processor -- start 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a413ac0 0x55f76a424520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a418d40 0x55f76a421ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a422730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a422e40 con 0x55f76a419660 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a482590 con 0x55f76a413ac0 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f76a4826d0 con 0x55f76a418d40 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa514a38b80 10 client.0 _ll_register_callbacks cb 0x7fa4f4009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa502d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a422730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.561+0000 7fa50259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a413ac0 0x55f76a424520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa502d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a422730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36426/0 (socket says 172.21.15.42:36426) 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa502d9d700 1 -- 172.21.15.42:0/2526096646 learned_addr learned my addr 172.21.15.42:0/2526096646 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:01:51.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa50259c700 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a413ac0 0x55f76a424520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39482/0 (socket says 172.21.15.42:39482) 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa501d9b700 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a418d40 0x55f76a421ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa50259c700 1 -- 172.21.15.42:0/2526096646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a418d40 msgr2=0x55f76a421ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa50259c700 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a418d40 0x55f76a421ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa50259c700 1 -- 172.21.15.42:0/2526096646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 msgr2=0x55f76a422730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa50259c700 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a422730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa50259c700 1 -- 172.21.15.42:0/2526096646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5041a6a30 con 0x55f76a413ac0 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa501d9b700 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a418d40 0x55f76a421ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa502d9d700 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a422730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa50259c700 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a413ac0 0x55f76a424520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55f76a42fd90 tx=0x7fa4f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd51e256e0e4e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa4e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/2526096646 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f4022040 con 0x55f76a413ac0 2023-12-12T10:01:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa514a38b80 1 -- 172.21.15.42:0/2526096646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4fc010040 con 0x55f76a413ac0 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.562+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/2526096646 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa4f40293f0 con 0x55f76a413ac0 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/2526096646 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4f4016070 con 0x55f76a413ac0 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa514a38b80 10 client.25541 fetch_fsmap learned FSMap version 31 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa514a38b80 1 -- 172.21.15.42:0/2526096646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fa4fc00a410 con 0x55f76a413ac0 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/2526096646 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4fc010040 con 0x55f76a413ac0 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/2526096646 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fa4f40050b0 con 0x55f76a413ac0 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa514a38b80 10 client.25541 fetch_fsmap finished waiting for FSMap version 31 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa514a38b80 10 client.25541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa514a38b80 20 client.25541 populate_metadata read hostname 'smithi042' 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f76a484010 0x55f76a486430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:01:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa514a38b80 4 client.25541 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:01:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa514a38b80 1 -- 172.21.15.42:0/2526096646 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f76a32df70 con 0x55f76a484010 2023-12-12T10:01:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.563+0000 7fa501d9b700 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f76a484010 0x55f76a486430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:01:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.564+0000 7fa4e9ffb700 10 client.25541 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:51.564+0000 7fa501d9b700 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f76a484010 0x55f76a486430 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84861f60f10a173 server_cookie=3266873088d19c84 in_seq=0 out_seq=0 2023-12-12T10:01:53.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:52 smithi042 ceph-mon[98894]: pgmap v100: 129 pgs: 129 active+clean; 72 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-12-12T10:01:53.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:52 smithi111 ceph-mon[102565]: pgmap v100: 129 pgs: 129 active+clean; 72 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-12-12T10:01:53.329 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:52 smithi173 ceph-mon[103308]: pgmap v100: 129 pgs: 129 active+clean; 72 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-12-12T10:01:54.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:54 smithi042 ceph-mon[98894]: pgmap v101: 129 pgs: 129 active+clean; 72 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-12-12T10:01:54.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:54 smithi111 ceph-mon[102565]: pgmap v101: 129 pgs: 129 active+clean; 72 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-12-12T10:01:54.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.760+0000 7fa4e9ffb700 1 -- 172.21.15.42:0/2526096646 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f76a32df70 con 0x55f76a484010 2023-12-12T10:01:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.760+0000 7fa4e9ffb700 10 client.25541 handle_command_reply: tid=0 2023-12-12T10:01:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.760+0000 7fa514a38b80 1 client.25541 shutdown 2023-12-12T10:01:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.760+0000 7fa514a38b80 1 -- 172.21.15.42:0/2526096646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a413ac0 msgr2=0x55f76a424520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.760+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a413ac0 0x55f76a424520 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55f76a42fd90 tx=0x7fa4f4004750 comp rx=0 tx=0).stop 2023-12-12T10:01:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.761+0000 7fa514a38b80 1 -- 172.21.15.42:0/2526096646 shutdown_connections 2023-12-12T10:01:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.761+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f76a484010 0x55f76a486430 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.761+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f76a413ac0 0x55f76a424520 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55f76a42fd90 tx=0x7fa4f4004750 comp rx=0 tx=0).stop 2023-12-12T10:01:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.761+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f76a419660 0x55f76a422730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.761+0000 7fa514a38b80 1 --2- 172.21.15.42:0/2526096646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f76a418d40 0x55f76a421ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.761+0000 7fa514a38b80 1 -- 172.21.15.42:0/2526096646 >> 172.21.15.42:0/2526096646 conn(0x55f76a342780 msgr2=0x55f76a408940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.761+0000 7fa4e9ffb700 0 client.25541 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:01:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.762+0000 7fa514a38b80 1 -- 172.21.15.42:0/2526096646 shutdown_connections 2023-12-12T10:01:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.762+0000 7fa514a38b80 1 -- 172.21.15.42:0/2526096646 wait complete. 2023-12-12T10:01:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.762+0000 7fa514a38b80 20 client.25541 trim_cache size 0 max 16384 2023-12-12T10:01:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:01:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:01:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:01:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:01:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 -- 172.21.15.42:0/3514006574 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa4ec0a15e0 msgr2=0x7fa4ec0a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa4ec0a15e0 0x7fa4ec0a3aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x55f76a40e8a0 tx=0x7fa4f4009260 comp rx=0 tx=0).stop 2023-12-12T10:01:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 -- 172.21.15.42:0/3514006574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa504105670 msgr2=0x7fa504196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:01:54.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa504105670 0x7fa504196a60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8004760 tx=0x7fa4f8004790 comp rx=0 tx=0).stop 2023-12-12T10:01:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 -- 172.21.15.42:0/3514006574 shutdown_connections 2023-12-12T10:01:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa504196fa0 0x7fa5041a5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa504100550 0x7fa504196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa4ec0a15e0 0x7fa4ec0a3aa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 --2- 172.21.15.42:0/3514006574 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa504105670 0x7fa504196a60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:01:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 -- 172.21.15.42:0/3514006574 >> 172.21.15.42:0/3514006574 conn(0x7fa5040fbdf0 msgr2=0x7fa5040fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:01:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 -- 172.21.15.42:0/3514006574 shutdown_connections 2023-12-12T10:01:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:01:54.763+0000 7fa50883f700 1 -- 172.21.15.42:0/3514006574 wait complete. 2023-12-12T10:01:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:54 smithi173 ceph-mon[103308]: pgmap v101: 129 pgs: 129 active+clean; 72 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 212 op/s 2023-12-12T10:01:55.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:55 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:01:55.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:55 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:01:55.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:55 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:01:56.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:56 smithi111 ceph-mon[102565]: pgmap v102: 129 pgs: 129 active+clean; 88 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 913 op/s 2023-12-12T10:01:56.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:56 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:56 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:56 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:56 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:56 smithi173 ceph-mon[103308]: pgmap v102: 129 pgs: 129 active+clean; 88 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 913 op/s 2023-12-12T10:01:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:56 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:56 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:56 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:56 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:56 smithi042 ceph-mon[98894]: pgmap v102: 129 pgs: 129 active+clean; 88 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 5.1 MiB/s wr, 913 op/s 2023-12-12T10:01:56.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:56 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:56 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:56 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:56.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:56 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:57.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:57 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:01:57.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:57 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:01:57.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:57 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:57.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:57 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:01:57.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:57 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:01:57.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:57 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:57.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:57 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:01:57.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:57 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:01:57.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:57 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:01:57.920 INFO:tasks.workunit.client.0.smithi042.stdout:created hierarchy, now cleaning up 2023-12-12T10:01:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:01:58 smithi173 ceph-mon[103308]: pgmap v103: 129 pgs: 129 active+clean; 88 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 871 op/s 2023-12-12T10:01:58.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:01:58 smithi042 ceph-mon[98894]: pgmap v103: 129 pgs: 129 active+clean; 88 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 871 op/s 2023-12-12T10:01:58.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:01:58 smithi111 ceph-mon[102565]: pgmap v103: 129 pgs: 129 active+clean; 88 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 871 op/s 2023-12-12T10:02:00.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:00 smithi173 ceph-mon[103308]: pgmap v104: 129 pgs: 129 active+clean; 71 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 2.04k op/s 2023-12-12T10:02:00.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:00 smithi042 ceph-mon[98894]: pgmap v104: 129 pgs: 129 active+clean; 71 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 2.04k op/s 2023-12-12T10:02:00.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:00 smithi111 ceph-mon[102565]: pgmap v104: 129 pgs: 129 active+clean; 71 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 2.04k op/s 2023-12-12T10:02:02.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:02 smithi173 ceph-mon[103308]: pgmap v105: 129 pgs: 129 active+clean; 71 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 1.89k op/s 2023-12-12T10:02:02.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:02 smithi042 ceph-mon[98894]: pgmap v105: 129 pgs: 129 active+clean; 71 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 1.89k op/s 2023-12-12T10:02:02.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:02 smithi111 ceph-mon[102565]: pgmap v105: 129 pgs: 129 active+clean; 71 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 1.89k op/s 2023-12-12T10:02:04.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:04 smithi173 ceph-mon[103308]: pgmap v106: 129 pgs: 129 active+clean; 71 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 1.90k op/s 2023-12-12T10:02:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:04 smithi042 ceph-mon[98894]: pgmap v106: 129 pgs: 129 active+clean; 71 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 1.90k op/s 2023-12-12T10:02:04.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:04 smithi111 ceph-mon[102565]: pgmap v106: 129 pgs: 129 active+clean; 71 MiB data, 1001 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 1.90k op/s 2023-12-12T10:02:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:06 smithi173 ceph-mon[103308]: pgmap v107: 129 pgs: 129 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 1.99k op/s 2023-12-12T10:02:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:06 smithi042 ceph-mon[98894]: pgmap v107: 129 pgs: 129 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 1.99k op/s 2023-12-12T10:02:06.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:06 smithi111 ceph-mon[102565]: pgmap v107: 129 pgs: 129 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.4 MiB/s wr, 1.99k op/s 2023-12-12T10:02:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.878+0000 7fea4e1c8700 1 -- 172.21.15.42:0/1777230046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea48100b50 msgr2=0x7fea48100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.878+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/1777230046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea48100b50 0x7fea48100fd0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fea30009990 tx=0x7fea30009ca0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.878+0000 7fea4e1c8700 1 -- 172.21.15.42:0/1777230046 shutdown_connections 2023-12-12T10:02:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.878+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/1777230046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea481053b0 0x7fea48105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.878+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/1777230046 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea48101510 0x7fea4810d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.878+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/1777230046 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea48100b50 0x7fea48100fd0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.878+0000 7fea4e1c8700 1 -- 172.21.15.42:0/1777230046 >> 172.21.15.42:0/1777230046 conn(0x7fea480769d0 msgr2=0x7fea480fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.878+0000 7fea4e1c8700 1 -- 172.21.15.42:0/1777230046 shutdown_connections 2023-12-12T10:02:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.878+0000 7fea4e1c8700 1 -- 172.21.15.42:0/1777230046 wait complete. 2023-12-12T10:02:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea4e1c8700 1 Processor -- start 2023-12-12T10:02:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea4e1c8700 1 -- start start 2023-12-12T10:02:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea4e1c8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea48100b50 0x7fea48198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea4e1c8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea48101510 0x7fea48198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea4e1c8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481053b0 0x7fea481a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea4e1c8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea48199620 con 0x7fea481053b0 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea4e1c8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea481a3e30 con 0x7fea48101510 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea4e1c8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea481a3f70 con 0x7fea48100b50 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea477fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea48100b50 0x7fea48198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea47fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481053b0 0x7fea481a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea477fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea48100b50 0x7fea48198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51422/0 (socket says 172.21.15.42:51422) 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea477fe700 1 -- 172.21.15.42:0/24294653 learned_addr learned my addr 172.21.15.42:0/24294653 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea47fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481053b0 0x7fea481a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36442/0 (socket says 172.21.15.42:36442) 2023-12-12T10:02:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.879+0000 7fea46ffd700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea48101510 0x7fea48198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea47fff700 1 -- 172.21.15.42:0/24294653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea48100b50 msgr2=0x7fea48198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea47fff700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea48100b50 0x7fea48198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea47fff700 1 -- 172.21.15.42:0/24294653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea48101510 msgr2=0x7fea48198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea47fff700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea48101510 0x7fea48198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea47fff700 1 -- 172.21.15.42:0/24294653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea300095f0 con 0x7fea481053b0 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea47fff700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481053b0 0x7fea481a38f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fea4000efc0 tx=0x7fea4000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea44ff9700 1 -- 172.21.15.42:0/24294653 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40018070 con 0x7fea481053b0 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea4e1c8700 1 -- 172.21.15.42:0/24294653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea481a41a0 con 0x7fea481053b0 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea44ff9700 1 -- 172.21.15.42:0/24294653 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea4000f040 con 0x7fea481053b0 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea44ff9700 1 -- 172.21.15.42:0/24294653 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000cc80 con 0x7fea481053b0 2023-12-12T10:02:07.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.880+0000 7fea4e1c8700 1 -- 172.21.15.42:0/24294653 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea481a46b0 con 0x7fea481053b0 2023-12-12T10:02:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.882+0000 7fea44ff9700 1 -- 172.21.15.42:0/24294653 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fea40003a00 con 0x7fea481053b0 2023-12-12T10:02:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.883+0000 7fea44ff9700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea340a15f0 0x7fea340a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.883+0000 7fea477fe700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea340a15f0 0x7fea340a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.883+0000 7fea44ff9700 1 -- 172.21.15.42:0/24294653 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fea400ccfd0 con 0x7fea481053b0 2023-12-12T10:02:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.883+0000 7fea477fe700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea340a15f0 0x7fea340a3ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea38009d20 tx=0x7fea38009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea5a3c1b80 1 Processor -- start 2023-12-12T10:02:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea5a3c1b80 1 -- start start 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b0e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b08580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 0x5623d1b08f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1a63af0 con 0x5623d1b0e660 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1a61410 con 0x5623d1b0dd40 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1a5cd00 con 0x5623d1b08ac0 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea477fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b0e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea46ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b08580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea477fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b0e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39506/0 (socket says 172.21.15.42:39506) 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea477fe700 1 -- 172.21.15.42:0/4164166707 learned_addr learned my addr 172.21.15.42:0/4164166707 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea47fff700 1 --2- 172.21.15.42:0/4164166707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 0x5623d1b08f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea477fe700 1 -- 172.21.15.42:0/4164166707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 msgr2=0x5623d1b08f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea477fe700 1 --2- 172.21.15.42:0/4164166707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 0x5623d1b08f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea477fe700 1 -- 172.21.15.42:0/4164166707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 msgr2=0x5623d1b08580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea477fe700 1 --2- 172.21.15.42:0/4164166707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b08580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea477fe700 1 -- 172.21.15.42:0/4164166707 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623d18d95d0 con 0x5623d1b0dd40 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.884+0000 7fea477fe700 1 --2- 172.21.15.42:0/4164166707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b0e120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea3800e540 tx=0x7fea38014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2706df29cc5a3d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea3dffb700 1 -- 172.21.15.42:0/4164166707 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38026430 con 0x5623d1b0dd40 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea3dffb700 1 -- 172.21.15.42:0/4164166707 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea38021070 con 0x5623d1b0dd40 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea3dffb700 1 -- 172.21.15.42:0/4164166707 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3801d6e0 con 0x5623d1b0dd40 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/4164166707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 msgr2=0x5623d1b0e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/4164166707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b0e120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fea3800e540 tx=0x7fea38014000 comp rx=0 tx=0).stop 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/4164166707 shutdown_connections 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/4164166707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b0e120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/4164166707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 0x5623d1b08f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/4164166707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b08580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/4164166707 >> 172.21.15.42:0/4164166707 conn(0x5623d1a37780 msgr2=0x5623d1b02c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/4164166707 shutdown_connections 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.885+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/4164166707 wait complete. 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea5a3c1b80 1 -- start start 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea5a3c1b80 1 Processor -- start 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b08ac0 0x5623d1b14640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0dd40 0x5623d1b119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b11fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1a16500 con 0x5623d1b0e660 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1a161a0 con 0x5623d1b08ac0 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d18db8c0 con 0x5623d1b0dd40 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea5a3c1b80 10 client.0 _ll_register_callbacks cb 0x5623d1941a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea47fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b11fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea46ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0dd40 0x5623d1b119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea46ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0dd40 0x5623d1b119d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51438/0 (socket says 172.21.15.42:51438) 2023-12-12T10:02:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea47fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b11fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36456/0 (socket says 172.21.15.42:36456) 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea46ffd700 1 -- 172.21.15.42:0/2617595857 learned_addr learned my addr 172.21.15.42:0/2617595857 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.886+0000 7fea477fe700 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b08ac0 0x5623d1b14640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea47fff700 1 -- 172.21.15.42:0/2617595857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0dd40 msgr2=0x5623d1b119d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea47fff700 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0dd40 0x5623d1b119d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea47fff700 1 -- 172.21.15.42:0/2617595857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b08ac0 msgr2=0x5623d1b14640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea47fff700 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b08ac0 0x5623d1b14640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea47fff700 1 -- 172.21.15.42:0/2617595857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea38017070 con 0x5623d1b0e660 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea46ffd700 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0dd40 0x5623d1b119d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea477fe700 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b08ac0 0x5623d1b14640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea47fff700 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b11fe0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fea400ca3d0 tx=0x7fea40099680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ece9bdb417ad19d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea26ffd700 1 -- 172.21.15.42:0/2617595857 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400b1070 con 0x5623d1b0e660 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2617595857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623d1afb730 con 0x5623d1b0e660 2023-12-12T10:02:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea26ffd700 1 -- 172.21.15.42:0/2617595857 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fea40099c50 con 0x5623d1b0e660 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea26ffd700 1 -- 172.21.15.42:0/2617595857 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea4009dda0 con 0x5623d1b0e660 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea5a3c1b80 10 client.15870 fetch_fsmap learned FSMap version 31 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2617595857 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5623d1885d80 con 0x5623d1b0e660 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.887+0000 7fea26ffd700 1 -- 172.21.15.42:0/2617595857 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea40063020 con 0x5623d1b0e660 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea26ffd700 1 -- 172.21.15.42:0/2617595857 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fea40099eb0 con 0x5623d1b0e660 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea5a3c1b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 31 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea5a3c1b80 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea5a3c1b80 20 client.15870 populate_metadata read hostname 'smithi042' 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5623d1b22090 0x5623d1b244b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea5a3c1b80 4 client.15870 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2617595857 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623d1ad7ee0 con 0x5623d1b22090 2023-12-12T10:02:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea477fe700 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5623d1b22090 0x5623d1b244b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea477fe700 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5623d1b22090 0x5623d1b244b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d38b41a76900bf9 server_cookie=763592fc15b53257 in_seq=0 out_seq=0 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.888+0000 7fea26ffd700 10 client.15870 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea26ffd700 1 -- 172.21.15.42:0/2617595857 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623d1ad7ee0 con 0x5623d1b22090 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea26ffd700 10 client.15870 handle_command_reply: tid=0 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 client.15870 shutdown 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2617595857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 msgr2=0x5623d1b11fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b11fe0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fea400ca3d0 tx=0x7fea40099680 comp rx=0 tx=0).stop 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2617595857 shutdown_connections 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5623d1b22090 0x5623d1b244b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b08ac0 0x5623d1b14640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b11fe0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea26ffd700 0 client.15870 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2617595857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0dd40 0x5623d1b119d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2617595857 >> 172.21.15.42:0/2617595857 conn(0x5623d1a37780 msgr2=0x5623d1b07570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2617595857 shutdown_connections 2023-12-12T10:02:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2617595857 wait complete. 2023-12-12T10:02:07.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.889+0000 7fea5a3c1b80 20 client.15870 trim_cache size 0 max 16384 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.904+0000 7fea5a3c1b80 1 Processor -- start 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.904+0000 7fea5a3c1b80 1 -- start start 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 0x5623d1b11f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b19040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b19580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1aff860 con 0x5623d1b0e660 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1b1f640 con 0x5623d1b0dd40 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1b08580 con 0x5623d1b08ac0 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea47fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b19580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea477fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 0x5623d1b11f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea46ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b19040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea47fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b19580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36464/0 (socket says 172.21.15.42:36464) 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea47fff700 1 -- 172.21.15.42:0/2992858791 learned_addr learned my addr 172.21.15.42:0/2992858791 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea477fe700 1 -- 172.21.15.42:0/2992858791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 msgr2=0x5623d1b19040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea477fe700 1 --2- 172.21.15.42:0/2992858791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b19040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea477fe700 1 -- 172.21.15.42:0/2992858791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 msgr2=0x5623d1b19580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea477fe700 1 --2- 172.21.15.42:0/2992858791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b19580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea477fe700 1 -- 172.21.15.42:0/2992858791 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623d1b09170 con 0x5623d1b08ac0 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea47fff700 1 --2- 172.21.15.42:0/2992858791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b19580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea477fe700 1 --2- 172.21.15.42:0/2992858791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 0x5623d1b11f60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea3800eb30 tx=0x7fea38005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80cf1635343bd9fd server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.905+0000 7fea3e7fc700 1 -- 172.21.15.42:0/2992858791 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3800ee60 con 0x5623d1b08ac0 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea3e7fc700 1 -- 172.21.15.42:0/2992858791 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea38005500 con 0x5623d1b08ac0 2023-12-12T10:02:07.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea3e7fc700 1 -- 172.21.15.42:0/2992858791 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38004de0 con 0x5623d1b08ac0 2023-12-12T10:02:07.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2992858791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 msgr2=0x5623d1b11f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2992858791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 0x5623d1b11f60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea3800eb30 tx=0x7fea38005ce0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2992858791 shutdown_connections 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2992858791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b0e660 0x5623d1b19580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2992858791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b08ac0 0x5623d1b11f60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/2992858791 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b19040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2992858791 >> 172.21.15.42:0/2992858791 conn(0x5623d1a37780 msgr2=0x5623d1b1daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2992858791 shutdown_connections 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/2992858791 wait complete. 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.906+0000 7fea5a3c1b80 1 -- start start 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea5a3c1b80 1 Processor -- start 2023-12-12T10:02:07.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b08ac0 0x5623d1b17020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b17560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea5a3c1b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0e660 0x5623d1b17aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1b7b960 con 0x5623d1b08ac0 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1b7baa0 con 0x5623d1b0dd40 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea5a3c1b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623d1b7bbe0 con 0x5623d1b0e660 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea5a3c1b80 10 client.0 _ll_register_callbacks cb 0x5623d1941a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b08ac0 0x5623d1b17020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea477fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b08ac0 0x5623d1b17020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36466/0 (socket says 172.21.15.42:36466) 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea477fe700 1 -- 172.21.15.42:0/3321259496 learned_addr learned my addr 172.21.15.42:0/3321259496 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea46ffd700 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b17560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea47fff700 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0e660 0x5623d1b17aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea46ffd700 1 -- 172.21.15.42:0/3321259496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0e660 msgr2=0x5623d1b17aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea46ffd700 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0e660 0x5623d1b17aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea46ffd700 1 -- 172.21.15.42:0/3321259496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b08ac0 msgr2=0x5623d1b17020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea46ffd700 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b08ac0 0x5623d1b17020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea46ffd700 1 -- 172.21.15.42:0/3321259496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623d1885d80 con 0x5623d1b0dd40 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea477fe700 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b08ac0 0x5623d1b17020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.907+0000 7fea47fff700 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0e660 0x5623d1b17aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea46ffd700 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b17560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5623d1b02a30 tx=0x7fea3000bac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01241343ca736d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea26ffd700 1 -- 172.21.15.42:0/3321259496 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3001d030 con 0x5623d1b0dd40 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea26ffd700 1 -- 172.21.15.42:0/3321259496 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fea30004910 con 0x5623d1b0dd40 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/3321259496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea38021070 con 0x5623d1b0dd40 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea26ffd700 1 -- 172.21.15.42:0/3321259496 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea30004af0 con 0x5623d1b0dd40 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea5a3c1b80 10 client.25559 fetch_fsmap learned FSMap version 31 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/3321259496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fea3802a040 con 0x5623d1b0dd40 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea26ffd700 1 -- 172.21.15.42:0/3321259496 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fea38021070 con 0x5623d1b0dd40 2023-12-12T10:02:07.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea26ffd700 1 -- 172.21.15.42:0/3321259496 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 31) v1 ==== 7906+0+0 (secure 0 0 0) 0x7fea3002fca0 con 0x5623d1b0dd40 2023-12-12T10:02:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea5a3c1b80 10 client.25559 fetch_fsmap finished waiting for FSMap version 31 2023-12-12T10:02:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea5a3c1b80 10 client.25559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:02:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea5a3c1b80 20 client.25559 populate_metadata read hostname 'smithi042' 2023-12-12T10:02:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5623d1b7d4b0 0x5623d1b7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea5a3c1b80 4 client.25559 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:02:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.908+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/3321259496 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5623d1a22f70 con 0x5623d1b7d4b0 2023-12-12T10:02:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.909+0000 7fea477fe700 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5623d1b7d4b0 0x5623d1b7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.909+0000 7fea26ffd700 10 client.25559 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:07.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:07.909+0000 7fea477fe700 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5623d1b7d4b0 0x5623d1b7f8d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=483c280729e077b7 server_cookie=f6be6322bfc71ec in_seq=0 out_seq=0 2023-12-12T10:02:08.828 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:08 smithi173 ceph-mon[103308]: pgmap v108: 129 pgs: 129 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 1.28k op/s 2023-12-12T10:02:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:08 smithi042 ceph-mon[98894]: pgmap v108: 129 pgs: 129 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 1.28k op/s 2023-12-12T10:02:08.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:08 smithi111 ceph-mon[102565]: pgmap v108: 129 pgs: 129 active+clean; 87 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 1.28k op/s 2023-12-12T10:02:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea26ffd700 1 -- 172.21.15.42:0/3321259496 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5623d1a22f70 con 0x5623d1b7d4b0 2023-12-12T10:02:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea26ffd700 10 client.25559 handle_command_reply: tid=0 2023-12-12T10:02:09.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea5a3c1b80 1 client.25559 shutdown 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/3321259496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 msgr2=0x5623d1b17560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b17560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5623d1b02a30 tx=0x7fea3000bac0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/3321259496 shutdown_connections 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5623d1b0e660 0x5623d1b17aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5623d1b08ac0 0x5623d1b17020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5623d1b7d4b0 0x5623d1b7f8d0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea5a3c1b80 1 --2- 172.21.15.42:0/3321259496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5623d1b0dd40 0x5623d1b17560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea26ffd700 0 client.25559 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.722+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/3321259496 >> 172.21.15.42:0/3321259496 conn(0x5623d1a37780 msgr2=0x5623d1b10220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.723+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/3321259496 shutdown_connections 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.723+0000 7fea5a3c1b80 1 -- 172.21.15.42:0/3321259496 wait complete. 2023-12-12T10:02:09.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.723+0000 7fea5a3c1b80 20 client.25559 trim_cache size 0 max 16384 2023-12-12T10:02:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:02:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:02:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:02:09.727 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.723+0000 7fea4e1c8700 1 -- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea340a15f0 msgr2=0x7fea340a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.723+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea340a15f0 0x7fea340a3ab0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fea38009d20 tx=0x7fea38009260 comp rx=0 tx=0).stop 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.723+0000 7fea4e1c8700 1 -- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481053b0 msgr2=0x7fea481a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.723+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481053b0 0x7fea481a38f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fea4000efc0 tx=0x7fea4000c3a0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.724+0000 7fea4e1c8700 1 -- 172.21.15.42:0/24294653 shutdown_connections 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.724+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fea481053b0 0x7fea481a38f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.724+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea48100b50 0x7fea48198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.724+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fea340a15f0 0x7fea340a3ab0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.724+0000 7fea4e1c8700 1 --2- 172.21.15.42:0/24294653 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea48101510 0x7fea48198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.724+0000 7fea4e1c8700 1 -- 172.21.15.42:0/24294653 >> 172.21.15.42:0/24294653 conn(0x7fea480769d0 msgr2=0x7fea480fc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.724+0000 7fea4e1c8700 1 -- 172.21.15.42:0/24294653 shutdown_connections 2023-12-12T10:02:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:09.724+0000 7fea4e1c8700 1 -- 172.21.15.42:0/24294653 wait complete. 2023-12-12T10:02:10.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:10 smithi173 ceph-mon[103308]: pgmap v109: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.6 MiB/s wr, 1.40k op/s 2023-12-12T10:02:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:10 smithi042 ceph-mon[98894]: pgmap v109: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.6 MiB/s wr, 1.40k op/s 2023-12-12T10:02:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:10 smithi111 ceph-mon[102565]: pgmap v109: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.6 MiB/s wr, 1.40k op/s 2023-12-12T10:02:12.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:12 smithi173 ceph-mon[103308]: pgmap v110: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 226 op/s 2023-12-12T10:02:12.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:12 smithi042 ceph-mon[98894]: pgmap v110: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 226 op/s 2023-12-12T10:02:12.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:12 smithi111 ceph-mon[102565]: pgmap v110: 129 pgs: 129 active+clean; 146 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 226 op/s 2023-12-12T10:02:14.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:14 smithi042 ceph-mon[98894]: pgmap v111: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 255 op/s 2023-12-12T10:02:14.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:14 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:14.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:14 smithi111 ceph-mon[102565]: pgmap v111: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 255 op/s 2023-12-12T10:02:14.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:14 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:15.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:14 smithi173 ceph-mon[103308]: pgmap v111: 129 pgs: 129 active+clean; 147 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 6.9 MiB/s wr, 255 op/s 2023-12-12T10:02:15.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:14 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:16.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:16 smithi042 ceph-mon[98894]: pgmap v112: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.3 MiB/s wr, 503 op/s 2023-12-12T10:02:16.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:16 smithi111 ceph-mon[102565]: pgmap v112: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.3 MiB/s wr, 503 op/s 2023-12-12T10:02:17.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:16 smithi173 ceph-mon[103308]: pgmap v112: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 9.3 MiB/s wr, 503 op/s 2023-12-12T10:02:17.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:17 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:17.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:17 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:18.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:17 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:18.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:18 smithi042 ceph-mon[98894]: pgmap v113: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.0 MiB/s wr, 408 op/s 2023-12-12T10:02:18.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:18 smithi111 ceph-mon[102565]: pgmap v113: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.0 MiB/s wr, 408 op/s 2023-12-12T10:02:19.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:18 smithi173 ceph-mon[103308]: pgmap v113: 129 pgs: 129 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.0 MiB/s wr, 408 op/s 2023-12-12T10:02:19.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:19 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:19.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:19 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:20.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:19 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:20.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:20 smithi042 ceph-mon[98894]: pgmap v114: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 762 op/s 2023-12-12T10:02:20.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:20 smithi111 ceph-mon[102565]: pgmap v114: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 762 op/s 2023-12-12T10:02:21.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:20 smithi173 ceph-mon[103308]: pgmap v114: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 762 op/s 2023-12-12T10:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.860+0000 7fd65ce9f700 1 -- 172.21.15.42:0/4103314138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658104420 msgr2=0x7fd6581048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:22.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.860+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/4103314138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658104420 0x7fd6581048a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009990 tx=0x7fd64c009ca0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.861+0000 7fd65ce9f700 1 -- 172.21.15.42:0/4103314138 shutdown_connections 2023-12-12T10:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.861+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/4103314138 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd658106460 0x7fd658103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.861+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/4103314138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd658068900 0x7fd658068d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.861+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/4103314138 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658104420 0x7fd6581048a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.861+0000 7fd65ce9f700 1 -- 172.21.15.42:0/4103314138 >> 172.21.15.42:0/4103314138 conn(0x7fd658077570 msgr2=0x7fd658077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.861+0000 7fd65ce9f700 1 -- 172.21.15.42:0/4103314138 shutdown_connections 2023-12-12T10:02:22.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.861+0000 7fd65ce9f700 1 -- 172.21.15.42:0/4103314138 wait complete. 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.862+0000 7fd65ce9f700 1 Processor -- start 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.862+0000 7fd65ce9f700 1 -- start start 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.862+0000 7fd65ce9f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd658068900 0x7fd65819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.862+0000 7fd65ce9f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd658104420 0x7fd65819e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65ce9f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658106460 0x7fd6581a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65ce9f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581a2e20 con 0x7fd658106460 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65ce9f700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581a2f90 con 0x7fd658068900 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65ce9f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6581a3150 con 0x7fd658104420 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd658068900 0x7fd65819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd656d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658106460 0x7fd6581a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd658068900 0x7fd65819df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39546/0 (socket says 172.21.15.42:39546) 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65659c700 1 -- 172.21.15.42:0/2194057801 learned_addr learned my addr 172.21.15.42:0/2194057801 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd655d9b700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd658104420 0x7fd65819e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65659c700 1 -- 172.21.15.42:0/2194057801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd658104420 msgr2=0x7fd65819e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65659c700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd658104420 0x7fd65819e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65659c700 1 -- 172.21.15.42:0/2194057801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658106460 msgr2=0x7fd6581a27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65659c700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658106460 0x7fd6581a27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65659c700 1 -- 172.21.15.42:0/2194057801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0095f0 con 0x7fd658068900 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65659c700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd658068900 0x7fd65819df00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e9a0 tx=0x7fd64800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd6477fe700 1 -- 172.21.15.42:0/2194057801 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648009790 con 0x7fd658068900 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd6477fe700 1 -- 172.21.15.42:0/2194057801 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd648010430 con 0x7fd658068900 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.863+0000 7fd65ce9f700 1 -- 172.21.15.42:0/2194057801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6581a3430 con 0x7fd658068900 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.864+0000 7fd6477fe700 1 -- 172.21.15.42:0/2194057801 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648018680 con 0x7fd658068900 2023-12-12T10:02:22.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.864+0000 7fd65ce9f700 1 -- 172.21.15.42:0/2194057801 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6581aac70 con 0x7fd658068900 2023-12-12T10:02:22.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.867+0000 7fd6477fe700 1 -- 172.21.15.42:0/2194057801 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fd6480105a0 con 0x7fd658068900 2023-12-12T10:02:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.868+0000 7fd6477fe700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd6400a15f0 0x7fd6400a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.868+0000 7fd655d9b700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd6400a15f0 0x7fd6400a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.868+0000 7fd6477fe700 1 -- 172.21.15.42:0/2194057801 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd648014070 con 0x7fd658068900 2023-12-12T10:02:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.868+0000 7fd669098b80 1 Processor -- start 2023-12-12T10:02:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.868+0000 7fd655d9b700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd6400a15f0 0x7fd6400a3ab0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00b2e0 tx=0x7fd64c005960 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:22.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd669098b80 1 -- start start 2023-12-12T10:02:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd669098b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62cf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd669098b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd669098b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd669098b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be6224af0 con 0x557be62cf660 2023-12-12T10:02:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd669098b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be6222410 con 0x557be62c9ac0 2023-12-12T10:02:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd669098b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be621dd00 con 0x557be62ced40 2023-12-12T10:02:22.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd655d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd655d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36494/0 (socket says 172.21.15.42:36494) 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd655d9b700 1 -- 172.21.15.42:0/2123171772 learned_addr learned my addr 172.21.15.42:0/2123171772 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd656d9d700 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.869+0000 7fd65659c700 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd656d9d700 1 -- 172.21.15.42:0/2123171772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 msgr2=0x557be62cf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd656d9d700 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62cf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd656d9d700 1 -- 172.21.15.42:0/2123171772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 msgr2=0x557be62c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd656d9d700 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd656d9d700 1 -- 172.21.15.42:0/2123171772 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557be609a5d0 con 0x557be62c9ac0 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd655d9b700 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:02:22.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd65659c700 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd656d9d700 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62c9f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd650011580 tx=0x7fd650009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dddec9a8239af848 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd644ff9700 1 -- 172.21.15.42:0/2123171772 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65000e040 con 0x557be62c9ac0 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd644ff9700 1 -- 172.21.15.42:0/2123171772 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd65000f040 con 0x557be62c9ac0 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.870+0000 7fd644ff9700 1 -- 172.21.15.42:0/2123171772 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6500044f0 con 0x557be62c9ac0 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.871+0000 7fd669098b80 1 -- 172.21.15.42:0/2123171772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 msgr2=0x557be62c9f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.871+0000 7fd669098b80 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62c9f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd650011580 tx=0x7fd650009920 comp rx=0 tx=0).stop 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.871+0000 7fd669098b80 1 -- 172.21.15.42:0/2123171772 shutdown_connections 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.871+0000 7fd669098b80 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62c9f00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.871+0000 7fd669098b80 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62cf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.871+0000 7fd669098b80 1 --2- 172.21.15.42:0/2123171772 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.871+0000 7fd669098b80 1 -- 172.21.15.42:0/2123171772 >> 172.21.15.42:0/2123171772 conn(0x557be61f8780 msgr2=0x557be62c3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.871+0000 7fd669098b80 1 -- 172.21.15.42:0/2123171772 shutdown_connections 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.871+0000 7fd669098b80 1 -- 172.21.15.42:0/2123171772 wait complete. 2023-12-12T10:02:22.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.872+0000 7fd669098b80 1 -- start start 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.872+0000 7fd669098b80 1 Processor -- start 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.872+0000 7fd669098b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62d55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.872+0000 7fd669098b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.872+0000 7fd669098b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62cf660 0x557be62d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.872+0000 7fd669098b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be61d7500 con 0x557be62ced40 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.872+0000 7fd669098b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be61d71a0 con 0x557be62c9ac0 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.872+0000 7fd669098b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be609c8c0 con 0x557be62cf660 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.872+0000 7fd669098b80 10 client.0 _ll_register_callbacks cb 0x557be6102a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd65659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62d55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd65659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62d55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39560/0 (socket says 172.21.15.42:39560) 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd65659c700 1 -- 172.21.15.42:0/2842898338 learned_addr learned my addr 172.21.15.42:0/2842898338 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd655d9b700 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd655d9b700 1 -- 172.21.15.42:0/2842898338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62cf660 msgr2=0x557be62d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd655d9b700 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62cf660 0x557be62d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd655d9b700 1 -- 172.21.15.42:0/2842898338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 msgr2=0x557be62d55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd655d9b700 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62d55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd655d9b700 1 -- 172.21.15.42:0/2842898338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6581aac70 con 0x557be62ced40 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd655d9b700 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62d28b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000c00 tx=0x7fd64c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8044af9239b4c551 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.873+0000 7fd63dffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd63dffb700 1 -- 172.21.15.42:0/2842898338 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c004c20 con 0x557be62ced40 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd63dffb700 1 -- 172.21.15.42:0/2842898338 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd64c004d80 con 0x557be62ced40 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd669098b80 1 -- 172.21.15.42:0/2842898338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557be62bc730 con 0x557be62ced40 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd63dffb700 1 -- 172.21.15.42:0/2842898338 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd64c00f460 con 0x557be62ced40 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd669098b80 10 client.15888 fetch_fsmap learned FSMap version 35 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd669098b80 1 -- 172.21.15.42:0/2842898338 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fd650011230 con 0x557be62ced40 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd63dffb700 1 -- 172.21.15.42:0/2842898338 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557be62bc730 con 0x557be62ced40 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd63dffb700 1 -- 172.21.15.42:0/2842898338 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 35) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fd64c030680 con 0x557be62ced40 2023-12-12T10:02:22.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd669098b80 10 client.15888 fetch_fsmap finished waiting for FSMap version 35 2023-12-12T10:02:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd669098b80 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:02:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd669098b80 20 client.15888 populate_metadata read hostname 'smithi042' 2023-12-12T10:02:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd669098b80 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557be62e3120 0x557be62e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd669098b80 4 client.15888 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:02:22.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.874+0000 7fd669098b80 1 -- 172.21.15.42:0/2842898338 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557be6298ee0 con 0x557be62e3120 2023-12-12T10:02:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.875+0000 7fd65659c700 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557be62e3120 0x557be62e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.875+0000 7fd63dffb700 10 client.15888 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.875+0000 7fd65659c700 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557be62e3120 0x557be62e5540 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c1f56954a74c96 server_cookie=5ae09c87b5083b21 in_seq=0 out_seq=0 2023-12-12T10:02:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd63dffb700 1 -- 172.21.15.42:0/2842898338 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557be6298ee0 con 0x557be62e3120 2023-12-12T10:02:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd63dffb700 10 client.15888 handle_command_reply: tid=0 2023-12-12T10:02:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 client.15888 shutdown 2023-12-12T10:02:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 -- 172.21.15.42:0/2842898338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 msgr2=0x557be62d28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62d28b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fd64c000c00 tx=0x7fd64c01a050 comp rx=0 tx=0).stop 2023-12-12T10:02:22.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 -- 172.21.15.42:0/2842898338 shutdown_connections 2023-12-12T10:02:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557be62e3120 0x557be62e5540 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62d55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62cf660 0x557be62d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 --2- 172.21.15.42:0/2842898338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62d28b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 -- 172.21.15.42:0/2842898338 >> 172.21.15.42:0/2842898338 conn(0x557be61f8780 msgr2=0x557be62c8570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd63dffb700 0 client.15888 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.876+0000 7fd669098b80 1 -- 172.21.15.42:0/2842898338 shutdown_connections 2023-12-12T10:02:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.877+0000 7fd669098b80 1 -- 172.21.15.42:0/2842898338 wait complete. 2023-12-12T10:02:22.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.877+0000 7fd669098b80 20 client.15888 trim_cache size 0 max 16384 2023-12-12T10:02:22.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.900+0000 7fd669098b80 1 Processor -- start 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.900+0000 7fd669098b80 1 -- start start 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.900+0000 7fd669098b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62c9ac0 0x557be62d2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.900+0000 7fd669098b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.900+0000 7fd669098b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62cf660 0x557be62da040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.900+0000 7fd669098b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be62bc4c0 con 0x557be62ced40 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.900+0000 7fd669098b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be62c0910 con 0x557be62cf660 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.900+0000 7fd669098b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be62bde40 con 0x557be62c9ac0 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd655d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd655d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62e9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36510/0 (socket says 172.21.15.42:36510) 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd655d9b700 1 -- 172.21.15.42:0/683188940 learned_addr learned my addr 172.21.15.42:0/683188940 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd65659c700 1 --2- 172.21.15.42:0/683188940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62c9ac0 0x557be62d2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd656d9d700 1 --2- 172.21.15.42:0/683188940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62cf660 0x557be62da040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:22 smithi042 ceph-mon[98894]: pgmap v115: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 641 op/s 2023-12-12T10:02:22.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:22 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd655d9b700 1 -- 172.21.15.42:0/683188940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62c9ac0 msgr2=0x557be62d2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd655d9b700 1 --2- 172.21.15.42:0/683188940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62c9ac0 0x557be62d2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd655d9b700 1 -- 172.21.15.42:0/683188940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62cf660 msgr2=0x557be62da040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd655d9b700 1 --2- 172.21.15.42:0/683188940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62cf660 0x557be62da040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd655d9b700 1 -- 172.21.15.42:0/683188940 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557be6046d80 con 0x557be62ced40 2023-12-12T10:02:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd655d9b700 1 --2- 172.21.15.42:0/683188940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62e9ea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd64c030a60 tx=0x7fd64c005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c560a7f58537c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.901+0000 7fd6457fa700 1 -- 172.21.15.42:0/683188940 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c009290 con 0x557be62ced40 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd6457fa700 1 -- 172.21.15.42:0/683188940 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd64c01e040 con 0x557be62ced40 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd6457fa700 1 -- 172.21.15.42:0/683188940 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c00f6c0 con 0x557be62ced40 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd669098b80 1 -- 172.21.15.42:0/683188940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 msgr2=0x557be62e9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd669098b80 1 --2- 172.21.15.42:0/683188940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62e9ea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fd64c030a60 tx=0x7fd64c005f00 comp rx=0 tx=0).stop 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd669098b80 1 -- 172.21.15.42:0/683188940 shutdown_connections 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd669098b80 1 --2- 172.21.15.42:0/683188940 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62cf660 0x557be62da040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd669098b80 1 --2- 172.21.15.42:0/683188940 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62c9ac0 0x557be62d2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd669098b80 1 --2- 172.21.15.42:0/683188940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62ced40 0x557be62e9ea0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd669098b80 1 -- 172.21.15.42:0/683188940 >> 172.21.15.42:0/683188940 conn(0x557be61f8780 msgr2=0x557be62c5530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.902+0000 7fd669098b80 1 -- 172.21.15.42:0/683188940 shutdown_connections 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.903+0000 7fd669098b80 1 -- 172.21.15.42:0/683188940 wait complete. 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.903+0000 7fd669098b80 1 -- start start 2023-12-12T10:02:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.903+0000 7fd669098b80 1 Processor -- start 2023-12-12T10:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.903+0000 7fd669098b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62da230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.903+0000 7fd669098b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62d8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.903+0000 7fd669098b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.903+0000 7fd669098b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be62d8aa0 con 0x557be62cf660 2023-12-12T10:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.903+0000 7fd669098b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be62d8c10 con 0x557be62c9ac0 2023-12-12T10:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.903+0000 7fd669098b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be62d8dd0 con 0x557be62ced40 2023-12-12T10:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd669098b80 10 client.0 _ll_register_callbacks cb 0x557be6102a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd656d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd655d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62d8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd655d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62d8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51500/0 (socket says 172.21.15.42:51500) 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd655d9b700 1 -- 172.21.15.42:0/2342329298 learned_addr learned my addr 172.21.15.42:0/2342329298 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd656d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62d8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36518/0 (socket says 172.21.15.42:36518) 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd656d9d700 1 -- 172.21.15.42:0/2342329298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 msgr2=0x557be62d8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd656d9d700 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62d8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd656d9d700 1 -- 172.21.15.42:0/2342329298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 msgr2=0x557be62da230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd656d9d700 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62da230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd656d9d700 1 -- 172.21.15.42:0/2342329298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd650011230 con 0x557be62cf660 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd656d9d700 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62d8560 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd650009b30 tx=0x7fd650009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38636fe72f2d5fe server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.904+0000 7fd63dffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:02:22.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd63dffb700 1 -- 172.21.15.42:0/2342329298 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65000e040 con 0x557be62cf660 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd63dffb700 1 -- 172.21.15.42:0/2342329298 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd650023070 con 0x557be62cf660 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd669098b80 1 -- 172.21.15.42:0/2342329298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd65000f040 con 0x557be62cf660 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd63dffb700 1 -- 172.21.15.42:0/2342329298 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd650007910 con 0x557be62cf660 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd669098b80 10 client.15894 fetch_fsmap learned FSMap version 35 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd669098b80 1 -- 172.21.15.42:0/2342329298 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x557be602fc00 con 0x557be62cf660 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd63dffb700 1 -- 172.21.15.42:0/2342329298 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd65000f040 con 0x557be62cf660 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd63dffb700 1 -- 172.21.15.42:0/2342329298 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 35) v1 ==== 7954+0+0 (secure 0 0 0) 0x7fd65001b020 con 0x557be62cf660 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd669098b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 35 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd669098b80 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd669098b80 20 client.15894 populate_metadata read hostname 'smithi042' 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd669098b80 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557be6339aa0 0x557be633bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd669098b80 4 client.15894 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.905+0000 7fd669098b80 1 -- 172.21.15.42:0/2342329298 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557be61e3f70 con 0x557be6339aa0 2023-12-12T10:02:22.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.906+0000 7fd65659c700 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557be6339aa0 0x557be633bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.906+0000 7fd63dffb700 10 client.15894 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:22.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:22.906+0000 7fd65659c700 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557be6339aa0 0x557be633bec0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8095a069bc8b70e server_cookie=4cac382a4a891ca in_seq=0 out_seq=0 2023-12-12T10:02:22.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:22 smithi111 ceph-mon[102565]: pgmap v115: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 641 op/s 2023-12-12T10:02:22.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:22 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:22 smithi173 ceph-mon[103308]: pgmap v115: 129 pgs: 129 active+clean; 180 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.9 MiB/s wr, 641 op/s 2023-12-12T10:02:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:22 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.523+0000 7fd63dffb700 1 -- 172.21.15.42:0/2342329298 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557be61e3f70 con 0x557be6339aa0 2023-12-12T10:02:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.523+0000 7fd63dffb700 10 client.15894 handle_command_reply: tid=0 2023-12-12T10:02:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.523+0000 7fd669098b80 1 client.15894 shutdown 2023-12-12T10:02:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 -- 172.21.15.42:0/2342329298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 msgr2=0x557be62d8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62d8560 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd650009b30 tx=0x7fd650009510 comp rx=0 tx=0).stop 2023-12-12T10:02:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 -- 172.21.15.42:0/2342329298 shutdown_connections 2023-12-12T10:02:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557be6339aa0 0x557be633bec0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557be62c9ac0 0x557be62da230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557be62cf660 0x557be62d8560 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd63dffb700 0 client.15894 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 --2- 172.21.15.42:0/2342329298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557be62ced40 0x557be62d8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 -- 172.21.15.42:0/2342329298 >> 172.21.15.42:0/2342329298 conn(0x557be61f8780 msgr2=0x557be62e9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 -- 172.21.15.42:0/2342329298 shutdown_connections 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.524+0000 7fd669098b80 1 -- 172.21.15.42:0/2342329298 wait complete. 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd669098b80 20 client.15894 trim_cache size 0 max 16384 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:02:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 -- 172.21.15.42:0/2194057801 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd6400a15f0 msgr2=0x7fd6400a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd6400a15f0 0x7fd6400a3ab0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd64c00b2e0 tx=0x7fd64c005960 comp rx=0 tx=0).stop 2023-12-12T10:02:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 -- 172.21.15.42:0/2194057801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd658068900 msgr2=0x7fd65819df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd658068900 0x7fd65819df00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd64800e9a0 tx=0x7fd64800ecb0 comp rx=0 tx=0).stop 2023-12-12T10:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 -- 172.21.15.42:0/2194057801 shutdown_connections 2023-12-12T10:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd6400a15f0 0x7fd6400a3ab0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd658068900 0x7fd65819df00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd658106460 0x7fd6581a27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 --2- 172.21.15.42:0/2194057801 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd658104420 0x7fd65819e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.525+0000 7fd65ce9f700 1 -- 172.21.15.42:0/2194057801 >> 172.21.15.42:0/2194057801 conn(0x7fd658077570 msgr2=0x7fd6580fe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.526+0000 7fd65ce9f700 1 -- 172.21.15.42:0/2194057801 shutdown_connections 2023-12-12T10:02:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:23.526+0000 7fd65ce9f700 1 -- 172.21.15.42:0/2194057801 wait complete. 2023-12-12T10:02:24.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:24 smithi042 ceph-mon[98894]: pgmap v116: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 675 op/s 2023-12-12T10:02:25.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:24 smithi111 ceph-mon[102565]: pgmap v116: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 675 op/s 2023-12-12T10:02:25.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:24 smithi173 ceph-mon[103308]: pgmap v116: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.3 MiB/s wr, 675 op/s 2023-12-12T10:02:26.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:26 smithi042 ceph-mon[98894]: pgmap v117: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.3 MiB/s wr, 950 op/s 2023-12-12T10:02:26.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:26 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:26 smithi111 ceph-mon[102565]: pgmap v117: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.3 MiB/s wr, 950 op/s 2023-12-12T10:02:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:26 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:27.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:26 smithi173 ceph-mon[103308]: pgmap v117: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.3 MiB/s wr, 950 op/s 2023-12-12T10:02:27.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:26 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:28.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:28 smithi042 ceph-mon[98894]: pgmap v118: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 692 op/s 2023-12-12T10:02:28.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:28 smithi111 ceph-mon[102565]: pgmap v118: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 692 op/s 2023-12-12T10:02:29.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:28 smithi173 ceph-mon[103308]: pgmap v118: 129 pgs: 129 active+clean; 205 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.9 MiB/s wr, 692 op/s 2023-12-12T10:02:30.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:30 smithi042 ceph-mon[98894]: pgmap v119: 129 pgs: 129 active+clean; 224 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 1.07k op/s 2023-12-12T10:02:30.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:30 smithi111 ceph-mon[102565]: pgmap v119: 129 pgs: 129 active+clean; 224 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 1.07k op/s 2023-12-12T10:02:31.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:30 smithi173 ceph-mon[103308]: pgmap v119: 129 pgs: 129 active+clean; 224 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 8.4 MiB/s wr, 1.07k op/s 2023-12-12T10:02:31.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:31 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:32.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:31 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:32.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:31 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:02:32.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:32 smithi111 ceph-mon[102565]: pgmap v120: 129 pgs: 129 active+clean; 224 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 714 op/s 2023-12-12T10:02:33.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:32 smithi173 ceph-mon[103308]: pgmap v120: 129 pgs: 129 active+clean; 224 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 714 op/s 2023-12-12T10:02:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:32 smithi042 ceph-mon[98894]: pgmap v120: 129 pgs: 129 active+clean; 224 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 714 op/s 2023-12-12T10:02:34.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:34 smithi111 ceph-mon[102565]: pgmap v121: 129 pgs: 129 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 742 op/s 2023-12-12T10:02:35.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:34 smithi173 ceph-mon[103308]: pgmap v121: 129 pgs: 129 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 742 op/s 2023-12-12T10:02:35.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:34 smithi042 ceph-mon[98894]: pgmap v121: 129 pgs: 129 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.8 MiB/s wr, 742 op/s 2023-12-12T10:02:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.711+0000 7f30a6dfa700 1 -- 172.21.15.42:0/181703091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30a0112250 msgr2=0x7f30a0114640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.711+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/181703091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30a0112250 0x7f30a0114640 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f309800b0a0 tx=0x7f309800b3b0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.714+0000 7f30a6dfa700 1 -- 172.21.15.42:0/181703091 shutdown_connections 2023-12-12T10:02:36.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.714+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/181703091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30a0112250 0x7f30a0114640 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.714+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/181703091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30a00ffdc0 0x7f30a01001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.714+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/181703091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a01006e0 0x7f30a0111d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.714+0000 7f30a6dfa700 1 -- 172.21.15.42:0/181703091 >> 172.21.15.42:0/181703091 conn(0x7f30a0075720 msgr2=0x7f30a0075b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.715+0000 7f30a6dfa700 1 -- 172.21.15.42:0/181703091 shutdown_connections 2023-12-12T10:02:36.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.715+0000 7f30a6dfa700 1 -- 172.21.15.42:0/181703091 wait complete. 2023-12-12T10:02:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.716+0000 7f30a6dfa700 1 Processor -- start 2023-12-12T10:02:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.716+0000 7f30a6dfa700 1 -- start start 2023-12-12T10:02:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.716+0000 7f30a6dfa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30a00ffdc0 0x7f30a019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.716+0000 7f30a6dfa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30a01006e0 0x7f30a019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.716+0000 7f30a6dfa700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a0112250 0x7f30a01a2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.716+0000 7f30a6dfa700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a01a2fc0 con 0x7f30a00ffdc0 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.716+0000 7f30a6dfa700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a01a3130 con 0x7f30a0112250 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.716+0000 7f30a6dfa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a01a32f0 con 0x7f30a01006e0 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f309ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30a01006e0 0x7f30a019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f309ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30a01006e0 0x7f30a019e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51514/0 (socket says 172.21.15.42:51514) 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f30a4b96700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30a00ffdc0 0x7f30a019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f309ffff700 1 -- 172.21.15.42:0/3207588133 learned_addr learned my addr 172.21.15.42:0/3207588133 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f30a5397700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a0112250 0x7f30a01a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f30a4b96700 1 -- 172.21.15.42:0/3207588133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30a01006e0 msgr2=0x7f30a019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f30a4b96700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30a01006e0 0x7f30a019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f30a4b96700 1 -- 172.21.15.42:0/3207588133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a0112250 msgr2=0x7f30a01a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f30a4b96700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a0112250 0x7f30a01a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f30a4b96700 1 -- 172.21.15.42:0/3207588133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3098009d00 con 0x7f30a00ffdc0 2023-12-12T10:02:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.717+0000 7f30a4b96700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30a00ffdc0 0x7f30a019e0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f309400b450 tx=0x7f309400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.718+0000 7f309dffb700 1 -- 172.21.15.42:0/3207588133 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094011830 con 0x7f30a00ffdc0 2023-12-12T10:02:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.718+0000 7f309dffb700 1 -- 172.21.15.42:0/3207588133 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309400be60 con 0x7f30a00ffdc0 2023-12-12T10:02:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.718+0000 7f309dffb700 1 -- 172.21.15.42:0/3207588133 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094010cc0 con 0x7f30a00ffdc0 2023-12-12T10:02:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.719+0000 7f30a6dfa700 1 -- 172.21.15.42:0/3207588133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a01a35d0 con 0x7f30a00ffdc0 2023-12-12T10:02:36.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.719+0000 7f30a6dfa700 1 -- 172.21.15.42:0/3207588133 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a01aae10 con 0x7f30a00ffdc0 2023-12-12T10:02:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.725+0000 7f30b2ff3b80 1 Processor -- start 2023-12-12T10:02:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.725+0000 7f309dffb700 1 -- 172.21.15.42:0/3207588133 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f3094010380 con 0x7f30a00ffdc0 2023-12-12T10:02:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.726+0000 7f309dffb700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f30880a16b0 0x7f30880a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.727+0000 7f309dffb700 1 -- 172.21.15.42:0/3207588133 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f30940ccc10 con 0x7f30a00ffdc0 2023-12-12T10:02:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.727+0000 7f30b2ff3b80 1 -- start start 2023-12-12T10:02:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.727+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f71d40 0x557ab9f72120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.727+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f6c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.727+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f6cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.727+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab9ec7af0 con 0x557ab9f71d40 2023-12-12T10:02:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.727+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab9ec5410 con 0x557ab9f6cac0 2023-12-12T10:02:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.727+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab9ec0d00 con 0x557ab9f72660 2023-12-12T10:02:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.728+0000 7f30a5397700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f30880a16b0 0x7f30880a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.728+0000 7f309ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f6c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.728+0000 7f309ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f6c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51524/0 (socket says 172.21.15.42:51524) 2023-12-12T10:02:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.728+0000 7f309ffff700 1 -- 172.21.15.42:0/2708570444 learned_addr learned my addr 172.21.15.42:0/2708570444 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.728+0000 7f30a5397700 1 --2- 172.21.15.42:0/2708570444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f6cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.728+0000 7f309ffff700 1 -- 172.21.15.42:0/2708570444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 msgr2=0x557ab9f6cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.728+0000 7f309ffff700 1 --2- 172.21.15.42:0/2708570444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f6cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.729+0000 7f309ffff700 1 -- 172.21.15.42:0/2708570444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f71d40 msgr2=0x557ab9f72120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.729+0000 7f309ffff700 1 --2- 172.21.15.42:0/2708570444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f71d40 0x557ab9f72120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.729+0000 7f309ffff700 1 -- 172.21.15.42:0/2708570444 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ab9d3d5d0 con 0x557ab9f72660 2023-12-12T10:02:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.729+0000 7f30a5397700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f30880a16b0 0x7f30880a3b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3098009fd0 tx=0x7f3098009340 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:36.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.729+0000 7f309ffff700 1 --2- 172.21.15.42:0/2708570444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f6c580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3090009b70 tx=0x7f3090009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b730f0817f3a9308 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.733+0000 7f3086ffd700 1 -- 172.21.15.42:0/2708570444 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090009670 con 0x557ab9f72660 2023-12-12T10:02:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.733+0000 7f3086ffd700 1 -- 172.21.15.42:0/2708570444 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309000ee50 con 0x557ab9f72660 2023-12-12T10:02:36.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.733+0000 7f3086ffd700 1 -- 172.21.15.42:0/2708570444 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090012630 con 0x557ab9f72660 2023-12-12T10:02:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.734+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/2708570444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 msgr2=0x557ab9f6c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.734+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/2708570444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f6c580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3090009b70 tx=0x7f3090009e80 comp rx=0 tx=0).stop 2023-12-12T10:02:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.734+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/2708570444 shutdown_connections 2023-12-12T10:02:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.734+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/2708570444 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f6cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.734+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/2708570444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f71d40 0x557ab9f72120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.734+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/2708570444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f6c580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.734+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/2708570444 >> 172.21.15.42:0/2708570444 conn(0x557ab9e9b780 msgr2=0x557ab9f66c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.735+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/2708570444 shutdown_connections 2023-12-12T10:02:36.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.735+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/2708570444 wait complete. 2023-12-12T10:02:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.736+0000 7f30b2ff3b80 1 -- start start 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.736+0000 7f30b2ff3b80 1 Processor -- start 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f75b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f71d40 0x557ab9f76050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f72660 0x557ab9f76590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab9e7a500 con 0x557ab9f72660 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab9e7a1a0 con 0x557ab9f6cac0 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab9d3f8c0 con 0x557ab9f71d40 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30b2ff3b80 10 client.0 _ll_register_callbacks cb 0x557ab9da5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30a5397700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f72660 0x557ab9f76590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30a5397700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f72660 0x557ab9f76590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36542/0 (socket says 172.21.15.42:36542) 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30a5397700 1 -- 172.21.15.42:0/3730881215 learned_addr learned my addr 172.21.15.42:0/3730881215 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.737+0000 7f30a5397700 1 -- 172.21.15.42:0/3730881215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f71d40 msgr2=0x557ab9f76050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:36.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.738+0000 7f30a5397700 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f71d40 0x557ab9f76050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.738+0000 7f30a5397700 1 -- 172.21.15.42:0/3730881215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 msgr2=0x557ab9f75b10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-12T10:02:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.738+0000 7f30a5397700 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f75b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.738+0000 7f30a5397700 1 -- 172.21.15.42:0/3730881215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30900097d0 con 0x557ab9f72660 2023-12-12T10:02:36.741 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.738+0000 7f30a5397700 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f72660 0x557ab9f76590 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3098003e20 tx=0x7f30980092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925183a301eed7bf server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:36.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.741+0000 7f307ffff700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:02:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.741+0000 7f307ffff700 1 -- 172.21.15.42:0/3730881215 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309800e040 con 0x557ab9f72660 2023-12-12T10:02:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.741+0000 7f307ffff700 1 -- 172.21.15.42:0/3730881215 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f309801e660 con 0x557ab9f72660 2023-12-12T10:02:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.741+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/3730881215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ab9f5f730 con 0x557ab9f72660 2023-12-12T10:02:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.742+0000 7f307ffff700 1 -- 172.21.15.42:0/3730881215 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f309801e820 con 0x557ab9f72660 2023-12-12T10:02:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.742+0000 7f30b2ff3b80 10 client.15903 fetch_fsmap learned FSMap version 37 2023-12-12T10:02:36.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.742+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/3730881215 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x557ab9ce9d80 con 0x557ab9f72660 2023-12-12T10:02:36.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.742+0000 7f307ffff700 1 -- 172.21.15.42:0/3730881215 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ab9f5f730 con 0x557ab9f72660 2023-12-12T10:02:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.744+0000 7f307ffff700 1 -- 172.21.15.42:0/3730881215 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f309802ba10 con 0x557ab9f72660 2023-12-12T10:02:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.744+0000 7f30b2ff3b80 10 client.15903 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:02:36.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.744+0000 7f30b2ff3b80 10 client.15903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:02:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.744+0000 7f30b2ff3b80 20 client.15903 populate_metadata read hostname 'smithi042' 2023-12-12T10:02:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.744+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557ab9f8ca70 0x557ab9f8ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.744+0000 7f30b2ff3b80 4 client.15903 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:02:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.744+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/3730881215 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ab9f3bee0 con 0x557ab9f8ca70 2023-12-12T10:02:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.746+0000 7f30a4b96700 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557ab9f8ca70 0x557ab9f8ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.748+0000 7f30a4b96700 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557ab9f8ca70 0x557ab9f8ee90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a2be4daa6cdabcf server_cookie=717fb1062680fa66 in_seq=0 out_seq=0 2023-12-12T10:02:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.748+0000 7f307ffff700 10 client.15903 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:36.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.750+0000 7f307ffff700 1 -- 172.21.15.42:0/3730881215 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557ab9f3bee0 con 0x557ab9f8ca70 2023-12-12T10:02:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.750+0000 7f307ffff700 10 client.15903 handle_command_reply: tid=0 2023-12-12T10:02:36.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.751+0000 7f30b2ff3b80 1 client.15903 shutdown 2023-12-12T10:02:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.757+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/3730881215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f72660 msgr2=0x557ab9f76590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.757+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f72660 0x557ab9f76590 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3098003e20 tx=0x7f30980092d0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.759+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/3730881215 shutdown_connections 2023-12-12T10:02:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.759+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557ab9f8ca70 0x557ab9f8ee90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.759+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f75b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.759+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f72660 0x557ab9f76590 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.759+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/3730881215 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f71d40 0x557ab9f76050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.759+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/3730881215 >> 172.21.15.42:0/3730881215 conn(0x557ab9e9b780 msgr2=0x557ab9f6b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.760+0000 7f307ffff700 0 client.15903 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.765+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/3730881215 shutdown_connections 2023-12-12T10:02:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.767+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/3730881215 wait complete. 2023-12-12T10:02:36.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.770+0000 7f30b2ff3b80 20 client.15903 trim_cache size 0 max 16384 2023-12-12T10:02:36.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.801+0000 7f30b2ff3b80 1 Processor -- start 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.802+0000 7f30b2ff3b80 1 -- start start 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.802+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f75050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.802+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f71d40 0x557ab9f93830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.802+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.802+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab9fa4c20 con 0x557ab9f71d40 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.802+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab9f77070 con 0x557ab9f6cac0 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.802+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ab9f5f700 con 0x557ab9f72660 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.803+0000 7f30a5397700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.803+0000 7f30a5397700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51534/0 (socket says 172.21.15.42:51534) 2023-12-12T10:02:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.803+0000 7f30a5397700 1 -- 172.21.15.42:0/4290328948 learned_addr learned my addr 172.21.15.42:0/4290328948 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.803+0000 7f30a5397700 1 -- 172.21.15.42:0/4290328948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 msgr2=0x557ab9f75050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-12T10:02:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.803+0000 7f30a5397700 1 --2- 172.21.15.42:0/4290328948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f75050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.803+0000 7f30a5397700 1 -- 172.21.15.42:0/4290328948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f71d40 msgr2=0x557ab9f93830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-12T10:02:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.803+0000 7f30a5397700 1 --2- 172.21.15.42:0/4290328948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f71d40 0x557ab9f93830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.803+0000 7f30a5397700 1 -- 172.21.15.42:0/4290328948 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ab9cd2c00 con 0x557ab9f72660 2023-12-12T10:02:36.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.803+0000 7f30a5397700 1 --2- 172.21.15.42:0/4290328948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9cd80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30980077c0 tx=0x7f3098007870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0c68f6214c85107 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.804+0000 7f30877fe700 1 -- 172.21.15.42:0/4290328948 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309800e040 con 0x557ab9f72660 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.804+0000 7f30877fe700 1 -- 172.21.15.42:0/4290328948 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309801e460 con 0x557ab9f72660 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.804+0000 7f30877fe700 1 -- 172.21.15.42:0/4290328948 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098035680 con 0x557ab9f72660 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.805+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/4290328948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 msgr2=0x557ab9f9cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.805+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/4290328948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9cd80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f30980077c0 tx=0x7f3098007870 comp rx=0 tx=0).stop 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.805+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/4290328948 shutdown_connections 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.805+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/4290328948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f6cac0 0x557ab9f75050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.805+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/4290328948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9cd80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.805+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/4290328948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ab9f71d40 0x557ab9f93830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.805+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/4290328948 >> 172.21.15.42:0/4290328948 conn(0x557ab9e9b780 msgr2=0x557ab9f6b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:36.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.807+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/4290328948 shutdown_connections 2023-12-12T10:02:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.812+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/4290328948 wait complete. 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.812+0000 7f30b2ff3b80 1 -- start start 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.812+0000 7f30b2ff3b80 1 Processor -- start 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f9b2a0 0x557ab9f9b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30b2ff3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557aba00ca00 0x557aba00edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aba00f330 con 0x557aba00ca00 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aba00f470 con 0x557ab9f9b2a0 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30b2ff3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aba00f600 con 0x557ab9f72660 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30b2ff3b80 10 client.0 _ll_register_callbacks cb 0x557ab9da5a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30a4b96700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30a4b96700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51542/0 (socket says 172.21.15.42:51542) 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f30a4b96700 1 -- 172.21.15.42:0/1738177548 learned_addr learned my addr 172.21.15.42:0/1738177548 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.813+0000 7f309ffff700 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f9b2a0 0x557ab9f9b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.814+0000 7f309ffff700 1 -- 172.21.15.42:0/1738177548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 msgr2=0x557ab9f9ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.814+0000 7f309ffff700 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.814+0000 7f309ffff700 1 -- 172.21.15.42:0/1738177548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557aba00ca00 msgr2=0x557aba00edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.814+0000 7f309ffff700 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557aba00ca00 0x557aba00edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.814+0000 7f309ffff700 1 -- 172.21.15.42:0/1738177548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ab9ce9d80 con 0x557ab9f9b2a0 2023-12-12T10:02:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.815+0000 7f309ffff700 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f9b2a0 0x557ab9f9b720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x557ab9f77040 tx=0x7f3090004320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b04b1b4ce4b10d48 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.815+0000 7f307ffff700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:02:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.816+0000 7f307ffff700 1 -- 172.21.15.42:0/1738177548 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090003940 con 0x557ab9f9b2a0 2023-12-12T10:02:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.816+0000 7f307ffff700 1 -- 172.21.15.42:0/1738177548 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3090003aa0 con 0x557ab9f9b2a0 2023-12-12T10:02:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.816+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/1738177548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f309000ee50 con 0x557ab9f9b2a0 2023-12-12T10:02:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.816+0000 7f307ffff700 1 -- 172.21.15.42:0/1738177548 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3090009650 con 0x557ab9f9b2a0 2023-12-12T10:02:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.817+0000 7f30b2ff3b80 10 client.25583 fetch_fsmap learned FSMap version 37 2023-12-12T10:02:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.817+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/1738177548 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x557ab9d101f0 con 0x557ab9f9b2a0 2023-12-12T10:02:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.820+0000 7f307ffff700 1 -- 172.21.15.42:0/1738177548 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309000ee50 con 0x557ab9f9b2a0 2023-12-12T10:02:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.820+0000 7f307ffff700 1 -- 172.21.15.42:0/1738177548 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f3090011070 con 0x557ab9f9b2a0 2023-12-12T10:02:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.820+0000 7f30b2ff3b80 10 client.25583 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:02:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.820+0000 7f30b2ff3b80 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:02:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.820+0000 7f30b2ff3b80 20 client.25583 populate_metadata read hostname 'smithi042' 2023-12-12T10:02:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.820+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557aba010d60 0x557aba013180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.820+0000 7f30b2ff3b80 4 client.25583 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:02:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.820+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/1738177548 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557ab9e86f70 con 0x557aba010d60 2023-12-12T10:02:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.824+0000 7f30a4b96700 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557aba010d60 0x557aba013180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.826+0000 7f30a4b96700 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557aba010d60 0x557aba013180 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=339c483e642595ce server_cookie=9304e8ca028d0abd in_seq=0 out_seq=0 2023-12-12T10:02:36.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:36.826+0000 7f307ffff700 10 client.25583 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:36.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:36 smithi111 ceph-mon[102565]: pgmap v122: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 1.02k op/s 2023-12-12T10:02:37.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:36 smithi173 ceph-mon[103308]: pgmap v122: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 1.02k op/s 2023-12-12T10:02:37.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:36 smithi042 ceph-mon[98894]: pgmap v122: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.7 MiB/s wr, 1.02k op/s 2023-12-12T10:02:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.488+0000 7f307ffff700 1 -- 172.21.15.42:0/1738177548 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557ab9e86f70 con 0x557aba010d60 2023-12-12T10:02:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.488+0000 7f307ffff700 10 client.25583 handle_command_reply: tid=0 2023-12-12T10:02:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.488+0000 7f30b2ff3b80 1 client.25583 shutdown 2023-12-12T10:02:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.489+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/1738177548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f9b2a0 msgr2=0x557ab9f9b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:37.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.489+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f9b2a0 0x557ab9f9b720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x557ab9f77040 tx=0x7f3090004320 comp rx=0 tx=0).stop 2023-12-12T10:02:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.489+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/1738177548 shutdown_connections 2023-12-12T10:02:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.489+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557aba00ca00 0x557aba00edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.489+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x557ab9f72660 0x557ab9f9ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.489+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x557aba010d60 0x557aba013180 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.489+0000 7f30b2ff3b80 1 --2- 172.21.15.42:0/1738177548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557ab9f9b2a0 0x557ab9f9b720 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.489+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/1738177548 >> 172.21.15.42:0/1738177548 conn(0x557ab9e9b780 msgr2=0x557ab9f74320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:37.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.490+0000 7f307ffff700 0 client.25583 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.490+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/1738177548 shutdown_connections 2023-12-12T10:02:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.490+0000 7f30b2ff3b80 1 -- 172.21.15.42:0/1738177548 wait complete. 2023-12-12T10:02:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:02:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:02:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:02:37.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:02:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.490+0000 7f30b2ff3b80 20 client.25583 trim_cache size 0 max 16384 2023-12-12T10:02:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.491+0000 7f30a6dfa700 1 -- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f30880a16b0 msgr2=0x7f30880a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.491+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f30880a16b0 0x7f30880a3b70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3098009fd0 tx=0x7f3098009340 comp rx=0 tx=0).stop 2023-12-12T10:02:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.491+0000 7f30a6dfa700 1 -- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30a00ffdc0 msgr2=0x7f30a019e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.491+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30a00ffdc0 0x7f30a019e0a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f309400b450 tx=0x7f309400b760 comp rx=0 tx=0).stop 2023-12-12T10:02:37.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.492+0000 7f30a6dfa700 1 -- 172.21.15.42:0/3207588133 shutdown_connections 2023-12-12T10:02:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.492+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f30880a16b0 0x7f30880a3b70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.492+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30a0112250 0x7f30a01a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.492+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30a00ffdc0 0x7f30a019e0a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.492+0000 7f30a6dfa700 1 --2- 172.21.15.42:0/3207588133 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30a01006e0 0x7f30a019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.492+0000 7f30a6dfa700 1 -- 172.21.15.42:0/3207588133 >> 172.21.15.42:0/3207588133 conn(0x7f30a0075720 msgr2=0x7f30a010fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.492+0000 7f30a6dfa700 1 -- 172.21.15.42:0/3207588133 shutdown_connections 2023-12-12T10:02:37.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:37.492+0000 7f30a6dfa700 1 -- 172.21.15.42:0/3207588133 wait complete. 2023-12-12T10:02:38.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:38 smithi111 ceph-mon[102565]: pgmap v123: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 717 op/s 2023-12-12T10:02:39.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:38 smithi173 ceph-mon[103308]: pgmap v123: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 717 op/s 2023-12-12T10:02:39.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:38 smithi042 ceph-mon[98894]: pgmap v123: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 5.2 MiB/s wr, 717 op/s 2023-12-12T10:02:40.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:40 smithi111 ceph-mon[102565]: pgmap v124: 129 pgs: 129 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.2 MiB/s wr, 1.14k op/s 2023-12-12T10:02:41.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:40 smithi173 ceph-mon[103308]: pgmap v124: 129 pgs: 129 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.2 MiB/s wr, 1.14k op/s 2023-12-12T10:02:41.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:40 smithi042 ceph-mon[98894]: pgmap v124: 129 pgs: 129 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.2 MiB/s wr, 1.14k op/s 2023-12-12T10:02:42.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:42 smithi111 ceph-mon[102565]: pgmap v125: 129 pgs: 129 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 767 op/s 2023-12-12T10:02:43.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:42 smithi173 ceph-mon[103308]: pgmap v125: 129 pgs: 129 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 767 op/s 2023-12-12T10:02:43.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:42 smithi042 ceph-mon[98894]: pgmap v125: 129 pgs: 129 active+clean; 261 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 767 op/s 2023-12-12T10:02:44.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:44 smithi111 ceph-mon[102565]: pgmap v126: 129 pgs: 129 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 795 op/s 2023-12-12T10:02:45.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:44 smithi173 ceph-mon[103308]: pgmap v126: 129 pgs: 129 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 795 op/s 2023-12-12T10:02:45.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:44 smithi042 ceph-mon[98894]: pgmap v126: 129 pgs: 129 active+clean; 267 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.2 MiB/s wr, 795 op/s 2023-12-12T10:02:46.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:46 smithi111 ceph-mon[102565]: pgmap v127: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 1.08k op/s 2023-12-12T10:02:47.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:46 smithi173 ceph-mon[103308]: pgmap v127: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 1.08k op/s 2023-12-12T10:02:47.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:46 smithi042 ceph-mon[98894]: pgmap v127: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.8 MiB/s wr, 1.08k op/s 2023-12-12T10:02:48.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:48 smithi111 ceph-mon[102565]: pgmap v128: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 770 op/s 2023-12-12T10:02:49.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:48 smithi173 ceph-mon[103308]: pgmap v128: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 770 op/s 2023-12-12T10:02:49.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:48 smithi042 ceph-mon[98894]: pgmap v128: 129 pgs: 129 active+clean; 279 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.5 MiB/s wr, 770 op/s 2023-12-12T10:02:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.674+0000 7efce9eda700 1 -- 172.21.15.42:0/1566894898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce4100d90 msgr2=0x7efce410d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.674+0000 7efce9eda700 1 --2- 172.21.15.42:0/1566894898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce4100d90 0x7efce410d2a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efcdc01c0d0 tx=0x7efcdc01c3e0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.674+0000 7efce9eda700 1 -- 172.21.15.42:0/1566894898 shutdown_connections 2023-12-12T10:02:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.674+0000 7efce9eda700 1 --2- 172.21.15.42:0/1566894898 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce4100d90 0x7efce410d2a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.674+0000 7efce9eda700 1 --2- 172.21.15.42:0/1566894898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce4106470 0x7efce4106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.674+0000 7efce9eda700 1 --2- 172.21.15.42:0/1566894898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce41003d0 0x7efce4100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.674+0000 7efce9eda700 1 -- 172.21.15.42:0/1566894898 >> 172.21.15.42:0/1566894898 conn(0x7efce40fbea0 msgr2=0x7efce40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.676+0000 7efce9eda700 1 -- 172.21.15.42:0/1566894898 shutdown_connections 2023-12-12T10:02:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.677+0000 7efce9eda700 1 -- 172.21.15.42:0/1566894898 wait complete. 2023-12-12T10:02:50.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.677+0000 7efce9eda700 1 Processor -- start 2023-12-12T10:02:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce9eda700 1 -- start start 2023-12-12T10:02:50.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce9eda700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce41003d0 0x7efce4108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce9eda700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce4100d90 0x7efce41091f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce9eda700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce4106470 0x7efce4193ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce9eda700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce4194080 con 0x7efce4106470 2023-12-12T10:02:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce9eda700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce41941f0 con 0x7efce41003d0 2023-12-12T10:02:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce9eda700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce41943b0 con 0x7efce4100d90 2023-12-12T10:02:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce4106470 0x7efce4193ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce4106470 0x7efce4193ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36570/0 (socket says 172.21.15.42:36570) 2023-12-12T10:02:50.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.678+0000 7efce3fff700 1 -- 172.21.15.42:0/2213534135 learned_addr learned my addr 172.21.15.42:0/2213534135 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:50.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.679+0000 7efce3fff700 1 -- 172.21.15.42:0/2213534135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce4100d90 msgr2=0x7efce41091f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-12T10:02:50.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.679+0000 7efce3fff700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce4100d90 0x7efce41091f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.679+0000 7efce3fff700 1 -- 172.21.15.42:0/2213534135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce41003d0 msgr2=0x7efce4108cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:50.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.679+0000 7efce3fff700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce41003d0 0x7efce4108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.679+0000 7efce3fff700 1 -- 172.21.15.42:0/2213534135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4009520 con 0x7efce4106470 2023-12-12T10:02:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.679+0000 7efce3fff700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce4106470 0x7efce4193ab0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efcdc01c7b0 tx=0x7efcdc0188b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.680+0000 7efce0ff9700 1 -- 172.21.15.42:0/2213534135 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc01f050 con 0x7efce4106470 2023-12-12T10:02:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.680+0000 7efce0ff9700 1 -- 172.21.15.42:0/2213534135 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcdc01de60 con 0x7efce4106470 2023-12-12T10:02:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.680+0000 7efce0ff9700 1 -- 172.21.15.42:0/2213534135 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc02c6b0 con 0x7efce4106470 2023-12-12T10:02:50.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.680+0000 7efce9eda700 1 -- 172.21.15.42:0/2213534135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc01ad10 con 0x7efce4106470 2023-12-12T10:02:50.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.681+0000 7efce9eda700 1 -- 172.21.15.42:0/2213534135 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce41948d0 con 0x7efce4106470 2023-12-12T10:02:50.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.684+0000 7efce0ff9700 1 -- 172.21.15.42:0/2213534135 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7efcdc005040 con 0x7efce4106470 2023-12-12T10:02:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.687+0000 7efcf60d3b80 1 Processor -- start 2023-12-12T10:02:50.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.687+0000 7efcf60d3b80 1 -- start start 2023-12-12T10:02:50.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7ef40 0x55ee78a7f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7f860 0x55ee78a8a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 0x55ee78a8cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee789d9af0 con 0x55ee78a8a740 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee789d7410 con 0x55ee78a7f860 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee789d2d00 con 0x55ee78a7ef40 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce2ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7ef40 0x55ee78a7f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce2ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7ef40 0x55ee78a7f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51560/0 (socket says 172.21.15.42:51560) 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce2ffd700 1 -- 172.21.15.42:0/3202877218 learned_addr learned my addr 172.21.15.42:0/3202877218 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce0ff9700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7efccc0a1740 0x7efccc0a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce3fff700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7efccc0a1740 0x7efccc0a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce2ffd700 1 -- 172.21.15.42:0/3202877218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7f860 msgr2=0x55ee78a8a200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce2ffd700 1 --2- 172.21.15.42:0/3202877218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7f860 0x55ee78a8a200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce2ffd700 1 -- 172.21.15.42:0/3202877218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 msgr2=0x55ee78a8cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:50.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce2ffd700 1 --2- 172.21.15.42:0/3202877218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 0x55ee78a8cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce2ffd700 1 -- 172.21.15.42:0/3202877218 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee7884f5d0 con 0x55ee78a7ef40 2023-12-12T10:02:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.688+0000 7efce0ff9700 1 -- 172.21.15.42:0/2213534135 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7efcdc0df560 con 0x7efce4106470 2023-12-12T10:02:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.689+0000 7efce3fff700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7efccc0a1740 0x7efccc0a3c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ac210 tx=0x7efcdc0b0040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.690+0000 7efce2ffd700 1 --2- 172.21.15.42:0/3202877218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7ef40 0x55ee78a7f320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efcd800ab80 tx=0x7efcd800ae90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=337f04846207d3ec server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.690+0000 7efcc9ffb700 1 -- 172.21.15.42:0/3202877218 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd800fb90 con 0x55ee78a7ef40 2023-12-12T10:02:50.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.690+0000 7efcc9ffb700 1 -- 172.21.15.42:0/3202877218 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd800fcf0 con 0x55ee78a7ef40 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.690+0000 7efcc9ffb700 1 -- 172.21.15.42:0/3202877218 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd80105d0 con 0x55ee78a7ef40 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.691+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3202877218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7ef40 msgr2=0x55ee78a7f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.691+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/3202877218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7ef40 0x55ee78a7f320 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efcd800ab80 tx=0x7efcd800ae90 comp rx=0 tx=0).stop 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.691+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3202877218 shutdown_connections 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.691+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/3202877218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 0x55ee78a8cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.691+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/3202877218 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7ef40 0x55ee78a7f320 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.691+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/3202877218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7f860 0x55ee78a8a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.691+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3202877218 >> 172.21.15.42:0/3202877218 conn(0x55ee789ad780 msgr2=0x55ee78a7ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.691+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3202877218 shutdown_connections 2023-12-12T10:02:50.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.691+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3202877218 wait complete. 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.692+0000 7efcf60d3b80 1 -- start start 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.692+0000 7efcf60d3b80 1 Processor -- start 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a7ef40 0x55ee78a8a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 0x55ee78a87400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a8a740 0x55ee78a87a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7898c500 con 0x55ee78a7ef40 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee7898c1a0 con 0x55ee78a8a740 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee788518c0 con 0x55ee78a7f860 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efcf60d3b80 10 client.0 _ll_register_callbacks cb 0x55ee788b7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efce2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a8a740 0x55ee78a87a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efce2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a8a740 0x55ee78a87a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39644/0 (socket says 172.21.15.42:39644) 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efce2ffd700 1 -- 172.21.15.42:0/2604308210 learned_addr learned my addr 172.21.15.42:0/2604308210 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efce37fe700 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 0x55ee78a87400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efce2ffd700 1 -- 172.21.15.42:0/2604308210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 msgr2=0x55ee78a87400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efce2ffd700 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 0x55ee78a87400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efce2ffd700 1 -- 172.21.15.42:0/2604308210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a7ef40 msgr2=0x55ee78a8a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efce2ffd700 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a7ef40 0x55ee78a8a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.693+0000 7efce2ffd700 1 -- 172.21.15.42:0/2604308210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd800a830 con 0x55ee78a8a740 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.694+0000 7efce2ffd700 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a8a740 0x55ee78a87a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efcd800b010 tx=0x7efcd800fc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a57cc23d2d781392 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.694+0000 7efcc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.694+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2604308210 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8011910 con 0x55ee78a8a740 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.694+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2604308210 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7efcd8011a70 con 0x55ee78a8a740 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.694+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2604308210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ee78a71730 con 0x55ee78a8a740 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.694+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2604308210 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcd801a450 con 0x55ee78a8a740 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.694+0000 7efcf60d3b80 10 client.25589 fetch_fsmap learned FSMap version 37 2023-12-12T10:02:50.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.694+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2604308210 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ee787fbd80 con 0x55ee78a8a740 2023-12-12T10:02:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.694+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2604308210 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ee78a71730 con 0x55ee78a8a740 2023-12-12T10:02:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.695+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2604308210 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7efcd80244c0 con 0x55ee78a8a740 2023-12-12T10:02:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.695+0000 7efcf60d3b80 10 client.25589 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:02:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.695+0000 7efcf60d3b80 10 client.25589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:02:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.695+0000 7efcf60d3b80 20 client.25589 populate_metadata read hostname 'smithi042' 2023-12-12T10:02:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.695+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ee78a9a2a0 0x55ee78a9c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.695+0000 7efcf60d3b80 4 client.25589 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:02:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.695+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2604308210 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ee78a4dee0 con 0x55ee78a9a2a0 2023-12-12T10:02:50.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.695+0000 7efce37fe700 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ee78a9a2a0 0x55ee78a9c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.696+0000 7efce37fe700 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ee78a9a2a0 0x55ee78a9c6c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=299ba72f2d8c0e90 server_cookie=15908e5834a0c6db in_seq=0 out_seq=0 2023-12-12T10:02:50.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.696+0000 7efcc2ffd700 10 client.25589 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.698+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2604308210 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ee78a4dee0 con 0x55ee78a9a2a0 2023-12-12T10:02:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.698+0000 7efcc2ffd700 10 client.25589 handle_command_reply: tid=0 2023-12-12T10:02:50.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.698+0000 7efcf60d3b80 1 client.25589 shutdown 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2604308210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a8a740 msgr2=0x55ee78a87a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a8a740 0x55ee78a87a10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7efcd800b010 tx=0x7efcd800fc10 comp rx=0 tx=0).stop 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2604308210 shutdown_connections 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ee78a9a2a0 0x55ee78a9c6c0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a8a740 0x55ee78a87a10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a7ef40 0x55ee78a8a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2604308210 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 0x55ee78a87400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2604308210 >> 172.21.15.42:0/2604308210 conn(0x55ee789ad780 msgr2=0x55ee78a81050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcc2ffd700 0 client.25589 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2604308210 shutdown_connections 2023-12-12T10:02:50.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.699+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2604308210 wait complete. 2023-12-12T10:02:50.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.700+0000 7efcf60d3b80 20 client.25589 trim_cache size 0 max 16384 2023-12-12T10:02:50.734 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efcf60d3b80 1 Processor -- start 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efcf60d3b80 1 -- start start 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a89dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a7f860 0x55ee78a95dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a8a740 0x55ee78a96300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee78a755d0 con 0x55ee78a7f860 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee78aa18b0 con 0x55ee78a7ef40 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee78a85950 con 0x55ee78a8a740 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efce37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a7f860 0x55ee78a95dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efce2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a89dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efce2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a89dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39650/0 (socket says 172.21.15.42:39650) 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efce2ffd700 1 -- 172.21.15.42:0/3389992620 learned_addr learned my addr 172.21.15.42:0/3389992620 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.725+0000 7efce2ffd700 1 --2- 172.21.15.42:0/3389992620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a8a740 0x55ee78a96300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.726+0000 7efce2ffd700 1 -- 172.21.15.42:0/3389992620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a8a740 msgr2=0x55ee78a96300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:50.735 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.726+0000 7efce2ffd700 1 --2- 172.21.15.42:0/3389992620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a8a740 0x55ee78a96300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.726+0000 7efce2ffd700 1 -- 172.21.15.42:0/3389992620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a7f860 msgr2=0x55ee78a95dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.726+0000 7efce2ffd700 1 --2- 172.21.15.42:0/3389992620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a7f860 0x55ee78a95dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.726+0000 7efce2ffd700 1 -- 172.21.15.42:0/3389992620 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ee787e4c00 con 0x55ee78a7ef40 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.726+0000 7efce2ffd700 1 --2- 172.21.15.42:0/3389992620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a89dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efcd800a680 tx=0x7efcd801f040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a16e0075f40af53d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.726+0000 7efcca7fc700 1 -- 172.21.15.42:0/3389992620 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8024ec0 con 0x55ee78a7ef40 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.726+0000 7efcca7fc700 1 -- 172.21.15.42:0/3389992620 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd8011bd0 con 0x55ee78a7ef40 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.726+0000 7efcca7fc700 1 -- 172.21.15.42:0/3389992620 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8005420 con 0x55ee78a7ef40 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3389992620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 msgr2=0x55ee78a89dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/3389992620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a89dd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efcd800a680 tx=0x7efcd801f040 comp rx=0 tx=0).stop 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3389992620 shutdown_connections 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/3389992620 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a89dd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/3389992620 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a8a740 0x55ee78a96300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/3389992620 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a7f860 0x55ee78a95dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3389992620 >> 172.21.15.42:0/3389992620 conn(0x55ee789ad780 msgr2=0x55ee78a8cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3389992620 shutdown_connections 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 -- 172.21.15.42:0/3389992620 wait complete. 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 -- start start 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.727+0000 7efcf60d3b80 1 Processor -- start 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a91620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 0x55ee78a8f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efcf60d3b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 0x55ee78a8fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee78a90080 con 0x55ee78a8a740 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee78a901f0 con 0x55ee78a7ef40 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efcf60d3b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ee78a903b0 con 0x55ee78a7f860 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efcf60d3b80 10 client.0 _ll_register_callbacks cb 0x55ee788b7a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:02:50.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a91620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 0x55ee78a8fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 0x55ee78a8fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36600/0 (socket says 172.21.15.42:36600) 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce37fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 0x55ee78a8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 -- 172.21.15.42:0/2781872507 learned_addr learned my addr 172.21.15.42:0/2781872507 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 -- 172.21.15.42:0/2781872507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 msgr2=0x55ee78a8f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 0x55ee78a8f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 -- 172.21.15.42:0/2781872507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 msgr2=0x55ee78a91620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a91620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 -- 172.21.15.42:0/2781872507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ee787fbd80 con 0x55ee78a8a740 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efce2ffd700 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 0x55ee78a8fb40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efcd800a380 tx=0x7efcd800afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37207a7b8e3f3d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.728+0000 7efcc2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:02:50.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2781872507 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8011710 con 0x55ee78a8a740 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2781872507 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7efcd8000e40 con 0x55ee78a8a740 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2781872507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcd800fcf0 con 0x55ee78a8a740 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2781872507 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcd8011870 con 0x55ee78a8a740 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcf60d3b80 10 client.15915 fetch_fsmap learned FSMap version 37 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2781872507 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55ee788221f0 con 0x55ee78a8a740 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2781872507 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcd800fcf0 con 0x55ee78a8a740 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2781872507 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7efcd800f360 con 0x55ee78a8a740 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcf60d3b80 10 client.15915 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcf60d3b80 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcf60d3b80 20 client.15915 populate_metadata read hostname 'smithi042' 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ee78af0900 0x55ee78af2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcf60d3b80 4 client.15915 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2781872507 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ee78998f70 con 0x55ee78af0900 2023-12-12T10:02:50.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.729+0000 7efce37fe700 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ee78af0900 0x55ee78af2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:02:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.730+0000 7efcc2ffd700 10 client.15915 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:50.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:50.730+0000 7efce37fe700 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ee78af0900 0x55ee78af2d20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dc4b2da026203ac server_cookie=9916d1f96e0dc732 in_seq=0 out_seq=0 2023-12-12T10:02:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:50 smithi111 ceph-mon[102565]: pgmap v129: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 1.07k op/s 2023-12-12T10:02:51.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:50 smithi173 ceph-mon[103308]: pgmap v129: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 1.07k op/s 2023-12-12T10:02:51.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:50 smithi042 ceph-mon[98894]: pgmap v129: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 5.0 MiB/s wr, 1.07k op/s 2023-12-12T10:02:51.753 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.750+0000 7efcc2ffd700 1 -- 172.21.15.42:0/2781872507 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ee78998f70 con 0x55ee78af0900 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.750+0000 7efcc2ffd700 10 client.15915 handle_command_reply: tid=0 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 client.15915 shutdown 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2781872507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 msgr2=0x55ee78a8fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 0x55ee78a8fb40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7efcd800a380 tx=0x7efcd800afd0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2781872507 shutdown_connections 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55ee78af0900 0x55ee78af2d20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55ee78a7ef40 0x55ee78a91620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcc2ffd700 0 client.15915 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ee78a8a740 0x55ee78a8fb40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.754 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 --2- 172.21.15.42:0/2781872507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ee78a7f860 0x55ee78a8f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2781872507 >> 172.21.15.42:0/2781872507 conn(0x55ee789ad780 msgr2=0x55ee78aa1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2781872507 shutdown_connections 2023-12-12T10:02:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 1 -- 172.21.15.42:0/2781872507 wait complete. 2023-12-12T10:02:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.751+0000 7efcf60d3b80 20 client.15915 trim_cache size 0 max 16384 2023-12-12T10:02:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:02:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:02:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:02:51.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:02:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 -- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7efccc0a1740 msgr2=0x7efccc0a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7efccc0a1740 0x7efccc0a3c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ac210 tx=0x7efcdc0b0040 comp rx=0 tx=0).stop 2023-12-12T10:02:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 -- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce4106470 msgr2=0x7efce4193ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:02:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce4106470 0x7efce4193ab0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efcdc01c7b0 tx=0x7efcdc0188b0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 -- 172.21.15.42:0/2213534135 shutdown_connections 2023-12-12T10:02:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7efccc0a1740 0x7efccc0a3c00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce41003d0 0x7efce4108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efce4106470 0x7efce4193ab0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 --2- 172.21.15.42:0/2213534135 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efce4100d90 0x7efce41091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:02:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.752+0000 7efce9eda700 1 -- 172.21.15.42:0/2213534135 >> 172.21.15.42:0/2213534135 conn(0x7efce40fbea0 msgr2=0x7efce40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:02:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.753+0000 7efce9eda700 1 -- 172.21.15.42:0/2213534135 shutdown_connections 2023-12-12T10:02:51.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:02:51.753+0000 7efce9eda700 1 -- 172.21.15.42:0/2213534135 wait complete. 2023-12-12T10:02:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:52 smithi111 ceph-mon[102565]: pgmap v130: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 640 op/s 2023-12-12T10:02:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:52 smithi173 ceph-mon[103308]: pgmap v130: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 640 op/s 2023-12-12T10:02:53.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:52 smithi042 ceph-mon[98894]: pgmap v130: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.0 MiB/s wr, 640 op/s 2023-12-12T10:02:54.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:54 smithi111 ceph-mon[102565]: pgmap v131: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 690 op/s 2023-12-12T10:02:55.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:54 smithi173 ceph-mon[103308]: pgmap v131: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 690 op/s 2023-12-12T10:02:55.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:54 smithi042 ceph-mon[98894]: pgmap v131: 129 pgs: 129 active+clean; 297 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.4 MiB/s wr, 690 op/s 2023-12-12T10:02:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:56 smithi111 ceph-mon[102565]: pgmap v132: 129 pgs: 129 active+clean; 312 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 1.17k op/s 2023-12-12T10:02:57.075 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:56 smithi173 ceph-mon[103308]: pgmap v132: 129 pgs: 129 active+clean; 312 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 1.17k op/s 2023-12-12T10:02:57.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:56 smithi042 ceph-mon[98894]: pgmap v132: 129 pgs: 129 active+clean; 312 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 1.17k op/s 2023-12-12T10:02:58.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:57 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:02:58.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:57 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:02:58.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:57 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:02:58.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:57 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:02:58.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:57 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:02:58.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:57 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:02:58.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:57 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:02:58.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:57 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:02:58.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:57 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:02:58.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:57 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:02:58.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:57 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:02:58.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:57 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:02:59.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:02:58 smithi173 ceph-mon[103308]: pgmap v133: 129 pgs: 129 active+clean; 312 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 851 op/s 2023-12-12T10:02:59.157 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:02:58 smithi042 ceph-mon[98894]: pgmap v133: 129 pgs: 129 active+clean; 312 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 851 op/s 2023-12-12T10:02:59.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:02:58 smithi111 ceph-mon[102565]: pgmap v133: 129 pgs: 129 active+clean; 312 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 3.6 MiB/s wr, 851 op/s 2023-12-12T10:03:01.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:00 smithi173 ceph-mon[103308]: pgmap v134: 129 pgs: 129 active+clean; 332 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.3 MiB/s wr, 1.22k op/s 2023-12-12T10:03:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:00 smithi042 ceph-mon[98894]: pgmap v134: 129 pgs: 129 active+clean; 332 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.3 MiB/s wr, 1.22k op/s 2023-12-12T10:03:01.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:00 smithi111 ceph-mon[102565]: pgmap v134: 129 pgs: 129 active+clean; 332 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.3 MiB/s wr, 1.22k op/s 2023-12-12T10:03:03.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:02 smithi173 ceph-mon[103308]: pgmap v135: 129 pgs: 129 active+clean; 332 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 921 op/s 2023-12-12T10:03:03.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:02 smithi042 ceph-mon[98894]: pgmap v135: 129 pgs: 129 active+clean; 332 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 921 op/s 2023-12-12T10:03:03.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:02 smithi111 ceph-mon[102565]: pgmap v135: 129 pgs: 129 active+clean; 332 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.8 MiB/s wr, 921 op/s 2023-12-12T10:03:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.957+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1112692891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5e811adb0 msgr2=0x7fd5e811d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.957+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1112692891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5e811adb0 0x7fd5e811d1a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000b0d0 tx=0x7fd5e000b3e0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.958+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1112692891 shutdown_connections 2023-12-12T10:03:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.958+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1112692891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e810fcf0 0x7fd5e81085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.958+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1112692891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5e811adb0 0x7fd5e811d1a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.958+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1112692891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5e8110160 0x7fd5e8108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.958+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1112692891 >> 172.21.15.42:0/1112692891 conn(0x7fd5e806c5a0 msgr2=0x7fd5e806c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.960+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1112692891 shutdown_connections 2023-12-12T10:03:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.960+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1112692891 wait complete. 2023-12-12T10:03:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.961+0000 7fd5ecc44700 1 Processor -- start 2023-12-12T10:03:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.961+0000 7fd5ecc44700 1 -- start start 2023-12-12T10:03:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.962+0000 7fd5ecc44700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5e811adb0 0x7fd5e81ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.962+0000 7fd5ecc44700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e81acdc0 0x7fd5e81a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.962+0000 7fd5ecc44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5e81a6ef0 0x7fd5e81a7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.962+0000 7fd5ecc44700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e81ad3c0 con 0x7fd5e81a6ef0 2023-12-12T10:03:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.962+0000 7fd5ecc44700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e8079830 con 0x7fd5e81acdc0 2023-12-12T10:03:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.962+0000 7fd5ecc44700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e80799f0 con 0x7fd5e811adb0 2023-12-12T10:03:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.964+0000 7fd5e5d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e81acdc0 0x7fd5e81a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.964+0000 7fd5e5d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e81acdc0 0x7fd5e81a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39670/0 (socket says 172.21.15.42:39670) 2023-12-12T10:03:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.964+0000 7fd5e5d9b700 1 -- 172.21.15.42:0/1401389137 learned_addr learned my addr 172.21.15.42:0/1401389137 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.964+0000 7fd5e5d9b700 1 -- 172.21.15.42:0/1401389137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5e811adb0 msgr2=0x7fd5e81ac880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-12T10:03:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.964+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5e811adb0 0x7fd5e81ac880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.964+0000 7fd5e5d9b700 1 -- 172.21.15.42:0/1401389137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5e81a6ef0 msgr2=0x7fd5e81a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.964+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5e81a6ef0 0x7fd5e81a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.964+0000 7fd5e5d9b700 1 -- 172.21.15.42:0/1401389137 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e0009d00 con 0x7fd5e81acdc0 2023-12-12T10:03:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.965+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e81acdc0 0x7fd5e81a69b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00b420 tx=0x7fd5dc00b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.965+0000 7fd5d77fe700 1 -- 172.21.15.42:0/1401389137 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc011830 con 0x7fd5e81acdc0 2023-12-12T10:03:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.965+0000 7fd5d77fe700 1 -- 172.21.15.42:0/1401389137 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5dc011e50 con 0x7fd5e81acdc0 2023-12-12T10:03:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.965+0000 7fd5d77fe700 1 -- 172.21.15.42:0/1401389137 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc010e70 con 0x7fd5e81acdc0 2023-12-12T10:03:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.966+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1401389137 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e8079cd0 con 0x7fd5e81acdc0 2023-12-12T10:03:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.966+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1401389137 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e807a160 con 0x7fd5e81acdc0 2023-12-12T10:03:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.971+0000 7fd5f8e3db80 1 Processor -- start 2023-12-12T10:03:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.971+0000 7fd5f8e3db80 1 -- start start 2023-12-12T10:03:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.972+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.972+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f8660 0x558ee75f2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.972+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f2ac0 0x558ee75f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.972+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee754daf0 con 0x558ee75f7d40 2023-12-12T10:03:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.972+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee754b410 con 0x558ee75f8660 2023-12-12T10:03:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.972+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7546d00 con 0x558ee75f2ac0 2023-12-12T10:03:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.972+0000 7fd5e6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.973+0000 7fd5e6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75f8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36614/0 (socket says 172.21.15.42:36614) 2023-12-12T10:03:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.973+0000 7fd5e6d9d700 1 -- 172.21.15.42:0/72873255 learned_addr learned my addr 172.21.15.42:0/72873255 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:04.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.973+0000 7fd5e659c700 1 --2- 172.21.15.42:0/72873255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f8660 0x558ee75f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:04.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.973+0000 7fd5e6d9d700 1 -- 172.21.15.42:0/72873255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f2ac0 msgr2=0x558ee75f2f00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-12T10:03:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.973+0000 7fd5e6d9d700 1 --2- 172.21.15.42:0/72873255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f2ac0 0x558ee75f2f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.973+0000 7fd5e6d9d700 1 -- 172.21.15.42:0/72873255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f8660 msgr2=0x558ee75f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.973+0000 7fd5e6d9d700 1 --2- 172.21.15.42:0/72873255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f8660 0x558ee75f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.973+0000 7fd5e6d9d700 1 -- 172.21.15.42:0/72873255 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ee73c35d0 con 0x558ee75f7d40 2023-12-12T10:03:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.974+0000 7fd5e6d9d700 1 --2- 172.21.15.42:0/72873255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75f8120 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000cf20 tx=0x7fd5e000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7044c828a650509c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:04.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.974+0000 7fd5d4ff9700 1 -- 172.21.15.42:0/72873255 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000e040 con 0x558ee75f7d40 2023-12-12T10:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.974+0000 7fd5d4ff9700 1 -- 172.21.15.42:0/72873255 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5e00092d0 con 0x558ee75f7d40 2023-12-12T10:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.975+0000 7fd5d4ff9700 1 -- 172.21.15.42:0/72873255 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00052d0 con 0x558ee75f7d40 2023-12-12T10:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.976+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/72873255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 msgr2=0x558ee75f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.976+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/72873255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75f8120 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000cf20 tx=0x7fd5e000cf50 comp rx=0 tx=0).stop 2023-12-12T10:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.977+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/72873255 shutdown_connections 2023-12-12T10:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.977+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/72873255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f2ac0 0x558ee75f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.977+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/72873255 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75f8120 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.977+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/72873255 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f8660 0x558ee75f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.977+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/72873255 >> 172.21.15.42:0/72873255 conn(0x558ee7521780 msgr2=0x558ee75ecc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.977+0000 7fd5d77fe700 1 -- 172.21.15.42:0/1401389137 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fd5dc010510 con 0x7fd5e81acdc0 2023-12-12T10:03:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.978+0000 7fd5d77fe700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd5d00a1580 0x7fd5d00a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.978+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/72873255 shutdown_connections 2023-12-12T10:03:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.978+0000 7fd5d77fe700 1 -- 172.21.15.42:0/1401389137 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd5dc0cccb0 con 0x7fd5e81acdc0 2023-12-12T10:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.980+0000 7fd5e659c700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd5d00a1580 0x7fd5d00a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.982+0000 7fd5e659c700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd5d00a1580 0x7fd5d00a3a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x558ee75ed8a0 tx=0x7fd5d8008040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.982+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/72873255 wait complete. 2023-12-12T10:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.983+0000 7fd5f8e3db80 1 -- start start 2023-12-12T10:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.983+0000 7fd5f8e3db80 1 Processor -- start 2023-12-12T10:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.983+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f2ac0 0x558ee75fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.983+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75fba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.983+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 0x558ee75fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7500500 con 0x558ee75f7d40 2023-12-12T10:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee75001a0 con 0x558ee75f2ac0 2023-12-12T10:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee73c58c0 con 0x558ee75f8660 2023-12-12T10:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5f8e3db80 10 client.0 _ll_register_callbacks cb 0x558ee742ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:03:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f2ac0 0x558ee75fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f2ac0 0x558ee75fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39680/0 (socket says 172.21.15.42:39680) 2023-12-12T10:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e659c700 1 -- 172.21.15.42:0/1075474050 learned_addr learned my addr 172.21.15.42:0/1075474050 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e659c700 1 -- 172.21.15.42:0/1075474050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 msgr2=0x558ee75fc1a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-12T10:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e659c700 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 0x558ee75fc1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e659c700 1 -- 172.21.15.42:0/1075474050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 msgr2=0x558ee75fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e659c700 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75fba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e659c700 1 -- 172.21.15.42:0/1075474050 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e811da30 con 0x558ee75f2ac0 2023-12-12T10:03:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.984+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.985+0000 7fd5e659c700 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f2ac0 0x558ee75fe690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8014bb0 tx=0x7fd5d80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=950781346b4b9b16 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.985+0000 7fd5cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.985+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1075474050 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8012b30 con 0x558ee75f2ac0 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.985+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1075474050 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ee75e5730 con 0x558ee75f2ac0 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.985+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1075474050 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd5d8012c90 con 0x558ee75f2ac0 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.985+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1075474050 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5d8012e90 con 0x558ee75f2ac0 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.985+0000 7fd5f8e3db80 10 client.25604 fetch_fsmap learned FSMap version 37 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.985+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1075474050 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558ee736fd80 con 0x558ee75f2ac0 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.986+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1075474050 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558ee75e5730 con 0x558ee75f2ac0 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.986+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1075474050 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fd5d80135d0 con 0x558ee75f2ac0 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.986+0000 7fd5f8e3db80 10 client.25604 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.986+0000 7fd5f8e3db80 10 client.25604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:03:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.986+0000 7fd5f8e3db80 20 client.25604 populate_metadata read hostname 'smithi042' 2023-12-12T10:03:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.987+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x558ee761d110 0x558ee761f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.987+0000 7fd5f8e3db80 4 client.25604 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:03:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.987+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1075474050 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ee75c1ee0 con 0x558ee761d110 2023-12-12T10:03:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.987+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x558ee761d110 0x558ee761f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.992+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x558ee761d110 0x558ee761f550 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79672a7b62913ef9 server_cookie=22e03b0ad308179b in_seq=0 out_seq=0 2023-12-12T10:03:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.992+0000 7fd5cdffb700 10 client.25604 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.993+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1075474050 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ee75c1ee0 con 0x558ee761d110 2023-12-12T10:03:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.993+0000 7fd5cdffb700 10 client.25604 handle_command_reply: tid=0 2023-12-12T10:03:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.993+0000 7fd5f8e3db80 1 client.25604 shutdown 2023-12-12T10:03:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.994+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1075474050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f2ac0 msgr2=0x558ee75fe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.994+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f2ac0 0x558ee75fe690 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8014bb0 tx=0x7fd5d80047b0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1075474050 shutdown_connections 2023-12-12T10:03:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x558ee761d110 0x558ee761f550 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f2ac0 0x558ee75fe690 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 0x558ee75fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1075474050 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f7d40 0x558ee75fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1075474050 >> 172.21.15.42:0/1075474050 conn(0x558ee7521780 msgr2=0x558ee75f1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5cdffb700 0 client.25604 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1075474050 shutdown_connections 2023-12-12T10:03:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1075474050 wait complete. 2023-12-12T10:03:04.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:04.995+0000 7fd5f8e3db80 20 client.25604 trim_cache size 0 max 16384 2023-12-12T10:03:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.024+0000 7fd5f8e3db80 1 Processor -- start 2023-12-12T10:03:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.024+0000 7fd5f8e3db80 1 -- start start 2023-12-12T10:03:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.024+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f2ac0 0x558ee75fb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.024+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 0x558ee7623eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.024+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 0x558ee7614080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.024+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee75ed8d0 con 0x558ee75f2ac0 2023-12-12T10:03:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.024+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee75e9860 con 0x558ee75f7d40 2023-12-12T10:03:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.024+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee75e95d0 con 0x558ee75f8660 2023-12-12T10:03:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.027+0000 7fd5e5d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 0x558ee7623eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.027+0000 7fd5e659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f2ac0 0x558ee75fb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.027+0000 7fd5e659c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f2ac0 0x558ee75fb340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36630/0 (socket says 172.21.15.42:36630) 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.027+0000 7fd5e659c700 1 -- 172.21.15.42:0/1660885371 learned_addr learned my addr 172.21.15.42:0/1660885371 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.027+0000 7fd5e6d9d700 1 --2- 172.21.15.42:0/1660885371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 0x558ee7614080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.028+0000 7fd5e6d9d700 1 -- 172.21.15.42:0/1660885371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 msgr2=0x558ee7623eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.028+0000 7fd5e6d9d700 1 --2- 172.21.15.42:0/1660885371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 0x558ee7623eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.028+0000 7fd5e6d9d700 1 -- 172.21.15.42:0/1660885371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f2ac0 msgr2=0x558ee75fb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.028+0000 7fd5e6d9d700 1 --2- 172.21.15.42:0/1660885371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f2ac0 0x558ee75fb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.028+0000 7fd5e6d9d700 1 -- 172.21.15.42:0/1660885371 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ee7358c00 con 0x558ee75f8660 2023-12-12T10:03:05.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.028+0000 7fd5e6d9d700 1 --2- 172.21.15.42:0/1660885371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 0x558ee7614080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000f80 tx=0x7fd5e0009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85c80e7b0b3f7553 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.040+0000 7fd5d57fa700 1 -- 172.21.15.42:0/1660885371 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e000e040 con 0x558ee75f8660 2023-12-12T10:03:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.040+0000 7fd5d57fa700 1 -- 172.21.15.42:0/1660885371 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5e00074a0 con 0x558ee75f8660 2023-12-12T10:03:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.040+0000 7fd5d57fa700 1 -- 172.21.15.42:0/1660885371 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e00053b0 con 0x558ee75f8660 2023-12-12T10:03:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.041+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1660885371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 msgr2=0x558ee7614080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.041+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1660885371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 0x558ee7614080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0000f80 tx=0x7fd5e0009710 comp rx=0 tx=0).stop 2023-12-12T10:03:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.042+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1660885371 shutdown_connections 2023-12-12T10:03:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.042+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1660885371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f8660 0x558ee7614080 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.042+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1660885371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f2ac0 0x558ee75fb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.042+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1660885371 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 0x558ee7623eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.042+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1660885371 >> 172.21.15.42:0/1660885371 conn(0x558ee7521780 msgr2=0x558ee75ecc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:05.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.045+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1660885371 shutdown_connections 2023-12-12T10:03:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.047+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1660885371 wait complete. 2023-12-12T10:03:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.047+0000 7fd5f8e3db80 1 -- start start 2023-12-12T10:03:05.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.047+0000 7fd5f8e3db80 1 Processor -- start 2023-12-12T10:03:05.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.048+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f2ac0 0x558ee7614080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.048+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 0x558ee76145c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.048+0000 7fd5f8e3db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f8660 0x558ee7612210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.048+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7612920 con 0x558ee75f8660 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.048+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7612a90 con 0x558ee75f7d40 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.048+0000 7fd5f8e3db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ee7612c50 con 0x558ee75f2ac0 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.049+0000 7fd5f8e3db80 10 client.0 _ll_register_callbacks cb 0x558ee742ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.050+0000 7fd5e6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f8660 0x558ee7612210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.050+0000 7fd5e6d9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f8660 0x558ee7612210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36634/0 (socket says 172.21.15.42:36634) 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.050+0000 7fd5e6d9d700 1 -- 172.21.15.42:0/1504143257 learned_addr learned my addr 172.21.15.42:0/1504143257 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:05.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.050+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 0x558ee76145c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.051+0000 7fd5e5d9b700 1 -- 172.21.15.42:0/1504143257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f2ac0 msgr2=0x558ee7614080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.051+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f2ac0 0x558ee7614080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.051+0000 7fd5e5d9b700 1 -- 172.21.15.42:0/1504143257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f8660 msgr2=0x558ee7612210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.051+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f8660 0x558ee7612210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.051+0000 7fd5e5d9b700 1 -- 172.21.15.42:0/1504143257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ee73c35d0 con 0x558ee75f7d40 2023-12-12T10:03:05.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.051+0000 7fd5e5d9b700 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 0x558ee76145c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0ca1c0 tx=0x7fd5dc010810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=145190129c4d7357 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.051+0000 7fd5cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:03:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.053+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1504143257 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5dc099ed0 con 0x558ee75f7d40 2023-12-12T10:03:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.053+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1504143257 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd5dc0a1e00 con 0x558ee75f7d40 2023-12-12T10:03:05.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.053+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1504143257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5e00092d0 con 0x558ee75f7d40 2023-12-12T10:03:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.053+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1504143257 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5dc0b2200 con 0x558ee75f7d40 2023-12-12T10:03:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.053+0000 7fd5f8e3db80 10 client.25610 fetch_fsmap learned FSMap version 37 2023-12-12T10:03:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.053+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1504143257 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558ee73961f0 con 0x558ee75f7d40 2023-12-12T10:03:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.053+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1504143257 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd5dc0b6020 con 0x558ee75f7d40 2023-12-12T10:03:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.054+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1504143257 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fd5dc0a1a00 con 0x558ee75f7d40 2023-12-12T10:03:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.055+0000 7fd5f8e3db80 10 client.25610 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:03:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.055+0000 7fd5f8e3db80 10 client.25610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:03:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.055+0000 7fd5f8e3db80 20 client.25610 populate_metadata read hostname 'smithi042' 2023-12-12T10:03:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.055+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x558ee7673b30 0x558ee7675f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.055+0000 7fd5f8e3db80 4 client.25610 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:03:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.055+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1504143257 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558ee750cf70 con 0x558ee7673b30 2023-12-12T10:03:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.055+0000 7fd5e659c700 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x558ee7673b30 0x558ee7675f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.056+0000 7fd5e659c700 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x558ee7673b30 0x558ee7675f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd1a35905317b9e server_cookie=addb5f86834e2fe7 in_seq=0 out_seq=0 2023-12-12T10:03:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.056+0000 7fd5cdffb700 10 client.25610 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:05.078 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:04 smithi173 ceph-mon[103308]: pgmap v136: 129 pgs: 129 active+clean; 336 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 950 op/s 2023-12-12T10:03:05.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:04 smithi042 ceph-mon[98894]: pgmap v136: 129 pgs: 129 active+clean; 336 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 950 op/s 2023-12-12T10:03:05.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:04 smithi111 ceph-mon[102565]: pgmap v136: 129 pgs: 129 active+clean; 336 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.0 MiB/s wr, 950 op/s 2023-12-12T10:03:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.652+0000 7fd5cdffb700 1 -- 172.21.15.42:0/1504143257 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558ee750cf70 con 0x558ee7673b30 2023-12-12T10:03:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.652+0000 7fd5cdffb700 10 client.25610 handle_command_reply: tid=0 2023-12-12T10:03:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.652+0000 7fd5f8e3db80 1 client.25610 shutdown 2023-12-12T10:03:05.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1504143257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 msgr2=0x558ee76145c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 0x558ee76145c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc0ca1c0 tx=0x7fd5dc010810 comp rx=0 tx=0).stop 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1504143257 shutdown_connections 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x558ee75f8660 0x558ee7612210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x558ee75f2ac0 0x558ee7614080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x558ee7673b30 0x558ee7675f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 --2- 172.21.15.42:0/1504143257 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x558ee75f7d40 0x558ee76145c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1504143257 >> 172.21.15.42:0/1504143257 conn(0x558ee7521780 msgr2=0x558ee7618b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5cdffb700 0 client.25610 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1504143257 shutdown_connections 2023-12-12T10:03:05.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.660+0000 7fd5f8e3db80 1 -- 172.21.15.42:0/1504143257 wait complete. 2023-12-12T10:03:05.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.661+0000 7fd5f8e3db80 20 client.25610 trim_cache size 0 max 16384 2023-12-12T10:03:05.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:03:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:03:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:03:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:03:05.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1401389137 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd5d00a1580 msgr2=0x7fd5d00a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd5d00a1580 0x7fd5d00a3a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x558ee75ed8a0 tx=0x7fd5d8008040 comp rx=0 tx=0).stop 2023-12-12T10:03:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1401389137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e81acdc0 msgr2=0x7fd5e81a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e81acdc0 0x7fd5e81a69b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc00b420 tx=0x7fd5dc00b7e0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1401389137 shutdown_connections 2023-12-12T10:03:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5e81a6ef0 0x7fd5e81a7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5e811adb0 0x7fd5e81ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd5d00a1580 0x7fd5d00a3a40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 --2- 172.21.15.42:0/1401389137 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5e81acdc0 0x7fd5e81a69b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.664+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1401389137 >> 172.21.15.42:0/1401389137 conn(0x7fd5e806c5a0 msgr2=0x7fd5e8118da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.665+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1401389137 shutdown_connections 2023-12-12T10:03:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:05.665+0000 7fd5ecc44700 1 -- 172.21.15.42:0/1401389137 wait complete. 2023-12-12T10:03:07.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:06 smithi173 ceph-mon[103308]: pgmap v137: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.5 MiB/s wr, 1.18k op/s 2023-12-12T10:03:07.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:06 smithi042 ceph-mon[98894]: pgmap v137: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.5 MiB/s wr, 1.18k op/s 2023-12-12T10:03:07.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:06 smithi111 ceph-mon[102565]: pgmap v137: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.5 MiB/s wr, 1.18k op/s 2023-12-12T10:03:09.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:08 smithi042 ceph-mon[98894]: pgmap v138: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 673 op/s 2023-12-12T10:03:09.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:08 smithi111 ceph-mon[102565]: pgmap v138: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 673 op/s 2023-12-12T10:03:09.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:08 smithi173 ceph-mon[103308]: pgmap v138: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 673 op/s 2023-12-12T10:03:11.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:10 smithi042 ceph-mon[98894]: pgmap v139: 129 pgs: 129 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 1.15k op/s 2023-12-12T10:03:11.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:10 smithi111 ceph-mon[102565]: pgmap v139: 129 pgs: 129 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 1.15k op/s 2023-12-12T10:03:11.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:10 smithi173 ceph-mon[103308]: pgmap v139: 129 pgs: 129 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 6.1 MiB/s wr, 1.15k op/s 2023-12-12T10:03:13.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:12 smithi042 ceph-mon[98894]: pgmap v140: 129 pgs: 129 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 782 op/s 2023-12-12T10:03:13.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:12 smithi111 ceph-mon[102565]: pgmap v140: 129 pgs: 129 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 782 op/s 2023-12-12T10:03:13.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:12 smithi173 ceph-mon[103308]: pgmap v140: 129 pgs: 129 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 782 op/s 2023-12-12T10:03:14.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:14 smithi042 ceph-mon[98894]: pgmap v141: 129 pgs: 129 active+clean; 377 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 826 op/s 2023-12-12T10:03:14.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:14 smithi111 ceph-mon[102565]: pgmap v141: 129 pgs: 129 active+clean; 377 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 826 op/s 2023-12-12T10:03:14.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:14 smithi173 ceph-mon[103308]: pgmap v141: 129 pgs: 129 active+clean; 377 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.9 MiB/s wr, 826 op/s 2023-12-12T10:03:16.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:16 smithi042 ceph-mon[98894]: pgmap v142: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.8 MiB/s wr, 1.12k op/s 2023-12-12T10:03:16.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:16 smithi111 ceph-mon[102565]: pgmap v142: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.8 MiB/s wr, 1.12k op/s 2023-12-12T10:03:16.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:16 smithi173 ceph-mon[103308]: pgmap v142: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.8 MiB/s wr, 1.12k op/s 2023-12-12T10:03:18.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:18 smithi042 ceph-mon[98894]: pgmap v143: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 838 op/s 2023-12-12T10:03:18.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:18 smithi111 ceph-mon[102565]: pgmap v143: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 838 op/s 2023-12-12T10:03:18.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:18 smithi173 ceph-mon[103308]: pgmap v143: 129 pgs: 129 active+clean; 391 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 838 op/s 2023-12-12T10:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.897+0000 7f48c5187700 1 -- 172.21.15.42:0/3395559814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c0110160 msgr2=0x7f48c0108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.897+0000 7f48c5187700 1 --2- 172.21.15.42:0/3395559814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c0110160 0x7f48c0108b10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b0a0 tx=0x7f48b800b3b0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.898+0000 7f48c5187700 1 -- 172.21.15.42:0/3395559814 shutdown_connections 2023-12-12T10:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.898+0000 7f48c5187700 1 --2- 172.21.15.42:0/3395559814 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48c011adb0 0x7f48c011d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.898+0000 7f48c5187700 1 --2- 172.21.15.42:0/3395559814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48c010fcf0 0x7f48c01085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.898+0000 7f48c5187700 1 --2- 172.21.15.42:0/3395559814 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c0110160 0x7f48c0108b10 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b0a0 tx=0x7f48b800b3b0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.898+0000 7f48c5187700 1 -- 172.21.15.42:0/3395559814 >> 172.21.15.42:0/3395559814 conn(0x7f48c006c5a0 msgr2=0x7f48c006c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.902+0000 7f48c5187700 1 -- 172.21.15.42:0/3395559814 shutdown_connections 2023-12-12T10:03:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.902+0000 7f48c5187700 1 -- 172.21.15.42:0/3395559814 wait complete. 2023-12-12T10:03:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.903+0000 7f48c5187700 1 Processor -- start 2023-12-12T10:03:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48c5187700 1 -- start start 2023-12-12T10:03:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48c5187700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c010fcf0 0x7f48c01194a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48c5187700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48c011adb0 0x7f48c0112570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48c5187700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48c0112ab0 0x7f48c0112f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48c5187700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0117030 con 0x7f48c011adb0 2023-12-12T10:03:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48c5187700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c01171a0 con 0x7f48c010fcf0 2023-12-12T10:03:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48c5187700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0117360 con 0x7f48c0112ab0 2023-12-12T10:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48bed9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c010fcf0 0x7f48c01194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48bed9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c010fcf0 0x7f48c01194a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39712/0 (socket says 172.21.15.42:39712) 2023-12-12T10:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48bed9d700 1 -- 172.21.15.42:0/3771722528 learned_addr learned my addr 172.21.15.42:0/3771722528 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:18.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.904+0000 7f48bf59e700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48c0112ab0 0x7f48c0112f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.905+0000 7f48bf59e700 1 -- 172.21.15.42:0/3771722528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c010fcf0 msgr2=0x7f48c01194a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.905+0000 7f48bf59e700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c010fcf0 0x7f48c01194a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.905+0000 7f48bf59e700 1 -- 172.21.15.42:0/3771722528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48c011adb0 msgr2=0x7f48c0112570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:18.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.905+0000 7f48bf59e700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48c011adb0 0x7f48c0112570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.905+0000 7f48bf59e700 1 -- 172.21.15.42:0/3771722528 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8009d00 con 0x7f48c0112ab0 2023-12-12T10:03:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.905+0000 7f48bf59e700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48c0112ab0 0x7f48c0112f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48c006d5f0 tx=0x7f48b000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.906+0000 7f48a7fff700 1 -- 172.21.15.42:0/3771722528 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b000cbd0 con 0x7f48c0112ab0 2023-12-12T10:03:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.906+0000 7f48c5187700 1 -- 172.21.15.42:0/3771722528 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c0117640 con 0x7f48c0112ab0 2023-12-12T10:03:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.906+0000 7f48c5187700 1 -- 172.21.15.42:0/3771722528 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c0117b20 con 0x7f48c0112ab0 2023-12-12T10:03:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.906+0000 7f48a7fff700 1 -- 172.21.15.42:0/3771722528 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48b000eeb0 con 0x7f48c0112ab0 2023-12-12T10:03:18.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.906+0000 7f48a7fff700 1 -- 172.21.15.42:0/3771722528 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0005020 con 0x7f48c0112ab0 2023-12-12T10:03:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.912+0000 7f48a7fff700 1 -- 172.21.15.42:0/3771722528 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f48b0005180 con 0x7f48c0112ab0 2023-12-12T10:03:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.913+0000 7f48d1380b80 1 Processor -- start 2023-12-12T10:03:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.913+0000 7f48a7fff700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48a80a1730 0x7f48a80a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.913+0000 7f48a7fff700 1 -- 172.21.15.42:0/3771722528 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f48b0014070 con 0x7f48c0112ab0 2023-12-12T10:03:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.914+0000 7f48be59c700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48a80a1730 0x7f48a80a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.916+0000 7f48be59c700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48a80a1730 0x7f48a80a3bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f48c006d410 tx=0x7f48b8006000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.920+0000 7f48d1380b80 1 -- start start 2023-12-12T10:03:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.920+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd17fd40 0x55b5dd180120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.920+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd180660 0x55b5dd17a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.920+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd17af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.920+0000 7f48d1380b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd0d5af0 con 0x55b5dd17aac0 2023-12-12T10:03:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.920+0000 7f48d1380b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd0d3410 con 0x55b5dd180660 2023-12-12T10:03:18.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.920+0000 7f48d1380b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd0ced00 con 0x55b5dd17fd40 2023-12-12T10:03:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.925+0000 7f48bf59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd17af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.925+0000 7f48bf59e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd17af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36662/0 (socket says 172.21.15.42:36662) 2023-12-12T10:03:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.925+0000 7f48bf59e700 1 -- 172.21.15.42:0/1794176043 learned_addr learned my addr 172.21.15.42:0/1794176043 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:18.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.926+0000 7f48be59c700 1 --2- 172.21.15.42:0/1794176043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd180660 0x55b5dd17a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.926+0000 7f48be59c700 1 -- 172.21.15.42:0/1794176043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd17fd40 msgr2=0x55b5dd180120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.926+0000 7f48be59c700 1 --2- 172.21.15.42:0/1794176043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd17fd40 0x55b5dd180120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.927+0000 7f48be59c700 1 -- 172.21.15.42:0/1794176043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 msgr2=0x55b5dd17af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.927+0000 7f48be59c700 1 --2- 172.21.15.42:0/1794176043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd17af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.927+0000 7f48be59c700 1 -- 172.21.15.42:0/1794176043 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5dcf4b5d0 con 0x55b5dd180660 2023-12-12T10:03:18.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.927+0000 7f48be59c700 1 --2- 172.21.15.42:0/1794176043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd180660 0x55b5dd17a580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48c0110560 tx=0x7f48b801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b70a4c15adec094e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.928+0000 7f48a57fa700 1 -- 172.21.15.42:0/1794176043 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b800e040 con 0x55b5dd180660 2023-12-12T10:03:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.928+0000 7f48a57fa700 1 -- 172.21.15.42:0/1794176043 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48b8008ed0 con 0x55b5dd180660 2023-12-12T10:03:18.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.928+0000 7f48a57fa700 1 -- 172.21.15.42:0/1794176043 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8027450 con 0x55b5dd180660 2023-12-12T10:03:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.929+0000 7f48d1380b80 1 -- 172.21.15.42:0/1794176043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd180660 msgr2=0x55b5dd17a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.929+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1794176043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd180660 0x55b5dd17a580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f48c0110560 tx=0x7f48b801a000 comp rx=0 tx=0).stop 2023-12-12T10:03:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.929+0000 7f48d1380b80 1 -- 172.21.15.42:0/1794176043 shutdown_connections 2023-12-12T10:03:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.929+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1794176043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd17af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.929+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1794176043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd17fd40 0x55b5dd180120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.929+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1794176043 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd180660 0x55b5dd17a580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.929+0000 7f48d1380b80 1 -- 172.21.15.42:0/1794176043 >> 172.21.15.42:0/1794176043 conn(0x55b5dd0a9780 msgr2=0x55b5dd174c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:18.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.929+0000 7f48d1380b80 1 -- 172.21.15.42:0/1794176043 shutdown_connections 2023-12-12T10:03:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.929+0000 7f48d1380b80 1 -- 172.21.15.42:0/1794176043 wait complete. 2023-12-12T10:03:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.930+0000 7f48d1380b80 1 -- start start 2023-12-12T10:03:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.930+0000 7f48d1380b80 1 Processor -- start 2023-12-12T10:03:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.930+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd17aac0 0x55b5dd1866a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.930+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17fd40 0x55b5dd183990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.930+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd180660 0x55b5dd184080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.930+0000 7f48d1380b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd088500 con 0x55b5dd180660 2023-12-12T10:03:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.930+0000 7f48d1380b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd0881a0 con 0x55b5dd17fd40 2023-12-12T10:03:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.930+0000 7f48d1380b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dcf4d8c0 con 0x55b5dd17aac0 2023-12-12T10:03:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.930+0000 7f48d1380b80 10 client.0 _ll_register_callbacks cb 0x55b5dcfb3a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:03:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.931+0000 7f48be59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17fd40 0x55b5dd183990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.931+0000 7f48be59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17fd40 0x55b5dd183990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39726/0 (socket says 172.21.15.42:39726) 2023-12-12T10:03:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.931+0000 7f48be59c700 1 -- 172.21.15.42:0/1147645652 learned_addr learned my addr 172.21.15.42:0/1147645652 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.931+0000 7f48be59c700 1 -- 172.21.15.42:0/1147645652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd17aac0 msgr2=0x55b5dd1866a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.931+0000 7f48be59c700 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd17aac0 0x55b5dd1866a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.931+0000 7f48be59c700 1 -- 172.21.15.42:0/1147645652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd180660 msgr2=0x55b5dd184080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.931+0000 7f48be59c700 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd180660 0x55b5dd184080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.931+0000 7f48be59c700 1 -- 172.21.15.42:0/1147645652 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c011da30 con 0x55b5dd17fd40 2023-12-12T10:03:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.931+0000 7f48be59c700 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17fd40 0x55b5dd183990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009a20 tx=0x7f48b800bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468dda3efbb4395 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.932+0000 7f48a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:03:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.932+0000 7f48a2ff5700 1 -- 172.21.15.42:0/1147645652 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b800e070 con 0x55b5dd17fd40 2023-12-12T10:03:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.932+0000 7f48a2ff5700 1 -- 172.21.15.42:0/1147645652 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f48b8020030 con 0x55b5dd17fd40 2023-12-12T10:03:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.932+0000 7f48d1380b80 1 -- 172.21.15.42:0/1147645652 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5dd16d730 con 0x55b5dd17fd40 2023-12-12T10:03:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.935+0000 7f48a2ff5700 1 -- 172.21.15.42:0/1147645652 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48b8003cc0 con 0x55b5dd17fd40 2023-12-12T10:03:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.935+0000 7f48d1380b80 10 client.25622 fetch_fsmap learned FSMap version 37 2023-12-12T10:03:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.935+0000 7f48d1380b80 1 -- 172.21.15.42:0/1147645652 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55b5dcef7d80 con 0x55b5dd17fd40 2023-12-12T10:03:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.941+0000 7f48a2ff5700 1 -- 172.21.15.42:0/1147645652 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b5dd16d730 con 0x55b5dd17fd40 2023-12-12T10:03:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.941+0000 7f48a2ff5700 1 -- 172.21.15.42:0/1147645652 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f48b803a470 con 0x55b5dd17fd40 2023-12-12T10:03:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.942+0000 7f48d1380b80 10 client.25622 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:03:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.942+0000 7f48d1380b80 10 client.25622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:03:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.942+0000 7f48d1380b80 20 client.25622 populate_metadata read hostname 'smithi042' 2023-12-12T10:03:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.942+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b5dd193d00 0x55b5dd196120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.942+0000 7f48d1380b80 4 client.25622 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:03:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.942+0000 7f48d1380b80 1 -- 172.21.15.42:0/1147645652 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5dd149ee0 con 0x55b5dd193d00 2023-12-12T10:03:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.949+0000 7f48bed9d700 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b5dd193d00 0x55b5dd196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.966+0000 7f48bed9d700 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b5dd193d00 0x55b5dd196120 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c2eda197f2a807 server_cookie=ad517d1acc963d2 in_seq=0 out_seq=0 2023-12-12T10:03:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.966+0000 7f48a2ff5700 10 client.25622 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.967+0000 7f48a2ff5700 1 -- 172.21.15.42:0/1147645652 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5dd149ee0 con 0x55b5dd193d00 2023-12-12T10:03:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.967+0000 7f48a2ff5700 10 client.25622 handle_command_reply: tid=0 2023-12-12T10:03:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48d1380b80 1 client.25622 shutdown 2023-12-12T10:03:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48d1380b80 1 -- 172.21.15.42:0/1147645652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17fd40 msgr2=0x55b5dd183990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17fd40 0x55b5dd183990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009a20 tx=0x7f48b800bd90 comp rx=0 tx=0).stop 2023-12-12T10:03:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48d1380b80 1 -- 172.21.15.42:0/1147645652 shutdown_connections 2023-12-12T10:03:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd180660 0x55b5dd184080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd17aac0 0x55b5dd1866a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b5dd193d00 0x55b5dd196120 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48d1380b80 1 --2- 172.21.15.42:0/1147645652 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17fd40 0x55b5dd183990 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009a20 tx=0x7f48b800bd90 comp rx=0 tx=0).stop 2023-12-12T10:03:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48a2ff5700 0 client.25622 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:18.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.968+0000 7f48d1380b80 1 -- 172.21.15.42:0/1147645652 >> 172.21.15.42:0/1147645652 conn(0x55b5dd0a9780 msgr2=0x55b5dd179570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.971+0000 7f48d1380b80 1 -- 172.21.15.42:0/1147645652 shutdown_connections 2023-12-12T10:03:18.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.982+0000 7f48d1380b80 1 -- 172.21.15.42:0/1147645652 wait complete. 2023-12-12T10:03:19.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:18.993+0000 7f48d1380b80 20 client.25622 trim_cache size 0 max 16384 2023-12-12T10:03:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.024+0000 7f48d1380b80 1 Processor -- start 2023-12-12T10:03:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.024+0000 7f48d1380b80 1 -- start start 2023-12-12T10:03:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.024+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17aac0 0x55b5dd18b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:19.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.024+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 0x55b5dd18b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd190200 0x55b5dd182280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48d1380b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd149ee0 con 0x55b5dd190200 2023-12-12T10:03:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48d1380b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd190ba0 con 0x55b5dd17aac0 2023-12-12T10:03:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48d1380b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd16d700 con 0x55b5dd180660 2023-12-12T10:03:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48be59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 0x55b5dd18b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48be59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 0x55b5dd18b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51658/0 (socket says 172.21.15.42:51658) 2023-12-12T10:03:19.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48be59c700 1 -- 172.21.15.42:0/143268235 learned_addr learned my addr 172.21.15.42:0/143268235 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48bf59e700 1 --2- 172.21.15.42:0/143268235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd190200 0x55b5dd182280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48be59c700 1 -- 172.21.15.42:0/143268235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17aac0 msgr2=0x55b5dd18b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48be59c700 1 --2- 172.21.15.42:0/143268235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17aac0 0x55b5dd18b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48be59c700 1 -- 172.21.15.42:0/143268235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd190200 msgr2=0x55b5dd182280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48be59c700 1 --2- 172.21.15.42:0/143268235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd190200 0x55b5dd182280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.025+0000 7f48be59c700 1 -- 172.21.15.42:0/143268235 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5dcee0c00 con 0x55b5dd180660 2023-12-12T10:03:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.027+0000 7f48be59c700 1 --2- 172.21.15.42:0/143268235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 0x55b5dd18b580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b0a0 tx=0x7f48b803abe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59fb8a933aa0ec09 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.037+0000 7f48a5ffb700 1 -- 172.21.15.42:0/143268235 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b800e070 con 0x55b5dd180660 2023-12-12T10:03:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.037+0000 7f48a5ffb700 1 -- 172.21.15.42:0/143268235 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48b8035070 con 0x55b5dd180660 2023-12-12T10:03:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.037+0000 7f48a5ffb700 1 -- 172.21.15.42:0/143268235 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b8005680 con 0x55b5dd180660 2023-12-12T10:03:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.043+0000 7f48d1380b80 1 -- 172.21.15.42:0/143268235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 msgr2=0x55b5dd18b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.043+0000 7f48d1380b80 1 --2- 172.21.15.42:0/143268235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 0x55b5dd18b580 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f48b800b0a0 tx=0x7f48b803abe0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.046+0000 7f48d1380b80 1 -- 172.21.15.42:0/143268235 shutdown_connections 2023-12-12T10:03:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.046+0000 7f48d1380b80 1 --2- 172.21.15.42:0/143268235 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd17aac0 0x55b5dd18b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.046+0000 7f48d1380b80 1 --2- 172.21.15.42:0/143268235 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd190200 0x55b5dd182280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.046+0000 7f48d1380b80 1 --2- 172.21.15.42:0/143268235 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 0x55b5dd18b580 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.046+0000 7f48d1380b80 1 -- 172.21.15.42:0/143268235 >> 172.21.15.42:0/143268235 conn(0x55b5dd0a9780 msgr2=0x55b5dd18faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.050+0000 7f48d1380b80 1 -- 172.21.15.42:0/143268235 shutdown_connections 2023-12-12T10:03:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.050+0000 7f48d1380b80 1 -- 172.21.15.42:0/143268235 wait complete. 2023-12-12T10:03:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.050+0000 7f48d1380b80 1 -- start start 2023-12-12T10:03:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.050+0000 7f48d1380b80 1 Processor -- start 2023-12-12T10:03:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.051+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd18ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.051+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 0x55b5dd18b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.051+0000 7f48d1380b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd190200 0x55b5dd189430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.051+0000 7f48d1380b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd189b10 con 0x55b5dd17aac0 2023-12-12T10:03:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.051+0000 7f48d1380b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd189c80 con 0x55b5dd190200 2023-12-12T10:03:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.051+0000 7f48d1380b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5dd189e40 con 0x55b5dd180660 2023-12-12T10:03:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.051+0000 7f48d1380b80 10 client.0 _ll_register_callbacks cb 0x7f48b0099d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:03:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.052+0000 7f48bed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd18ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.052+0000 7f48bed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd18ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36680/0 (socket says 172.21.15.42:36680) 2023-12-12T10:03:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.052+0000 7f48bed9d700 1 -- 172.21.15.42:0/192118457 learned_addr learned my addr 172.21.15.42:0/192118457 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.053+0000 7f48bed9d700 1 -- 172.21.15.42:0/192118457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 msgr2=0x55b5dd18b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.053+0000 7f48bed9d700 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 0x55b5dd18b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.053+0000 7f48bed9d700 1 -- 172.21.15.42:0/192118457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd190200 msgr2=0x55b5dd189430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.053+0000 7f48bed9d700 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd190200 0x55b5dd189430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.053+0000 7f48bed9d700 1 -- 172.21.15.42:0/192118457 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5dcef7d80 con 0x55b5dd17aac0 2023-12-12T10:03:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.053+0000 7f48bed9d700 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd18ade0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f48b4005830 tx=0x7f48b4012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a797a6090bcf52b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.055+0000 7f48a2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:03:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.055+0000 7f48a2ff5700 1 -- 172.21.15.42:0/192118457 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b4012db0 con 0x55b5dd17aac0 2023-12-12T10:03:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.056+0000 7f48a2ff5700 1 -- 172.21.15.42:0/192118457 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f48b4017070 con 0x55b5dd17aac0 2023-12-12T10:03:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.056+0000 7f48d1380b80 1 -- 172.21.15.42:0/192118457 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f48b8008ed0 con 0x55b5dd17aac0 2023-12-12T10:03:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.056+0000 7f48a2ff5700 1 -- 172.21.15.42:0/192118457 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48b4004030 con 0x55b5dd17aac0 2023-12-12T10:03:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.056+0000 7f48d1380b80 10 client.15939 fetch_fsmap learned FSMap version 37 2023-12-12T10:03:19.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.056+0000 7f48d1380b80 1 -- 172.21.15.42:0/192118457 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55b5dcf1e1f0 con 0x55b5dd17aac0 2023-12-12T10:03:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.057+0000 7f48a2ff5700 1 -- 172.21.15.42:0/192118457 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f48b8008ed0 con 0x55b5dd17aac0 2023-12-12T10:03:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.057+0000 7f48a2ff5700 1 -- 172.21.15.42:0/192118457 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f48b4003680 con 0x55b5dd17aac0 2023-12-12T10:03:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.057+0000 7f48d1380b80 10 client.15939 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:03:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.057+0000 7f48d1380b80 10 client.15939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:03:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.057+0000 7f48d1380b80 20 client.15939 populate_metadata read hostname 'smithi042' 2023-12-12T10:03:19.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.057+0000 7f48d1380b80 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b5dd1eb560 0x55b5dd1ed980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.057+0000 7f48d1380b80 4 client.15939 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:03:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.057+0000 7f48d1380b80 1 -- 172.21.15.42:0/192118457 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b5dd094f70 con 0x55b5dd1eb560 2023-12-12T10:03:19.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.060+0000 7f48be59c700 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b5dd1eb560 0x55b5dd1ed980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.061+0000 7f48be59c700 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b5dd1eb560 0x55b5dd1ed980 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3131c572b43800 server_cookie=29efa627414caa96 in_seq=0 out_seq=0 2023-12-12T10:03:19.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.061+0000 7f48a2ff5700 10 client.15939 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.424+0000 7f48a2ff5700 1 -- 172.21.15.42:0/192118457 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b5dd094f70 con 0x55b5dd1eb560 2023-12-12T10:03:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.424+0000 7f48a2ff5700 10 client.15939 handle_command_reply: tid=0 2023-12-12T10:03:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.424+0000 7f48d1380b80 1 client.15939 shutdown 2023-12-12T10:03:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.424+0000 7f48d1380b80 1 -- 172.21.15.42:0/192118457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 msgr2=0x55b5dd18ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.424+0000 7f48d1380b80 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd18ade0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f48b4005830 tx=0x7f48b4012730 comp rx=0 tx=0).stop 2023-12-12T10:03:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.425+0000 7f48d1380b80 1 -- 172.21.15.42:0/192118457 shutdown_connections 2023-12-12T10:03:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.425+0000 7f48d1380b80 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55b5dd1eb560 0x55b5dd1ed980 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.425+0000 7f48d1380b80 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55b5dd190200 0x55b5dd189430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.425+0000 7f48d1380b80 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5dd17aac0 0x55b5dd18ade0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.425+0000 7f48d1380b80 1 --2- 172.21.15.42:0/192118457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b5dd180660 0x55b5dd18b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.425+0000 7f48d1380b80 1 -- 172.21.15.42:0/192118457 >> 172.21.15.42:0/192118457 conn(0x55b5dd0a9780 msgr2=0x55b5dd182210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.425+0000 7f48a2ff5700 0 client.15939 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:19.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.428+0000 7f48d1380b80 1 -- 172.21.15.42:0/192118457 shutdown_connections 2023-12-12T10:03:19.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.431+0000 7f48d1380b80 1 -- 172.21.15.42:0/192118457 wait complete. 2023-12-12T10:03:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48d1380b80 20 client.15939 trim_cache size 0 max 16384 2023-12-12T10:03:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:03:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:03:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:03:19.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:03:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 -- 172.21.15.42:0/3771722528 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48a80a1730 msgr2=0x7f48a80a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48a80a1730 0x7f48a80a3bf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f48c006d410 tx=0x7f48b8006000 comp rx=0 tx=0).stop 2023-12-12T10:03:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 -- 172.21.15.42:0/3771722528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48c0112ab0 msgr2=0x7f48c0112f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48c0112ab0 0x7f48c0112f60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48c006d5f0 tx=0x7f48b000ecd0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 -- 172.21.15.42:0/3771722528 shutdown_connections 2023-12-12T10:03:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48a80a1730 0x7f48a80a3bf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f48c010fcf0 0x7f48c01194a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48c0112ab0 0x7f48c0112f60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 --2- 172.21.15.42:0/3771722528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f48c011adb0 0x7f48c0112570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.432+0000 7f48c5187700 1 -- 172.21.15.42:0/3771722528 >> 172.21.15.42:0/3771722528 conn(0x7f48c006c5a0 msgr2=0x7f48c006ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.437+0000 7f48c5187700 1 -- 172.21.15.42:0/3771722528 shutdown_connections 2023-12-12T10:03:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:19.437+0000 7f48c5187700 1 -- 172.21.15.42:0/3771722528 wait complete. 2023-12-12T10:03:20.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:20 smithi042 ceph-mon[98894]: pgmap v144: 129 pgs: 129 active+clean; 411 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 1.16k op/s 2023-12-12T10:03:20.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:20 smithi111 ceph-mon[102565]: pgmap v144: 129 pgs: 129 active+clean; 411 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 1.16k op/s 2023-12-12T10:03:20.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:20 smithi173 ceph-mon[103308]: pgmap v144: 129 pgs: 129 active+clean; 411 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.6 MiB/s wr, 1.16k op/s 2023-12-12T10:03:22.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:22 smithi111 ceph-mon[102565]: pgmap v145: 129 pgs: 129 active+clean; 411 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 681 op/s 2023-12-12T10:03:22.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:22 smithi173 ceph-mon[103308]: pgmap v145: 129 pgs: 129 active+clean; 411 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 681 op/s 2023-12-12T10:03:22.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:22 smithi042 ceph-mon[98894]: pgmap v145: 129 pgs: 129 active+clean; 411 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.4 MiB/s wr, 681 op/s 2023-12-12T10:03:24.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:24 smithi111 ceph-mon[102565]: pgmap v146: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 716 op/s 2023-12-12T10:03:24.827 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:24 smithi173 ceph-mon[103308]: pgmap v146: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 716 op/s 2023-12-12T10:03:24.910 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:24 smithi042 ceph-mon[98894]: pgmap v146: 129 pgs: 129 active+clean; 409 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.6 MiB/s wr, 716 op/s 2023-12-12T10:03:26.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:26 smithi111 ceph-mon[102565]: pgmap v147: 129 pgs: 129 active+clean; 422 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 1.08k op/s 2023-12-12T10:03:26.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:26 smithi173 ceph-mon[103308]: pgmap v147: 129 pgs: 129 active+clean; 422 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 1.08k op/s 2023-12-12T10:03:26.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:26 smithi042 ceph-mon[98894]: pgmap v147: 129 pgs: 129 active+clean; 422 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.8 MiB/s wr, 1.08k op/s 2023-12-12T10:03:28.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:28 smithi111 ceph-mon[102565]: pgmap v148: 129 pgs: 129 active+clean; 422 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 760 op/s 2023-12-12T10:03:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:28 smithi173 ceph-mon[103308]: pgmap v148: 129 pgs: 129 active+clean; 422 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 760 op/s 2023-12-12T10:03:28.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:28 smithi042 ceph-mon[98894]: pgmap v148: 129 pgs: 129 active+clean; 422 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.7 MiB/s wr, 760 op/s 2023-12-12T10:03:30.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:30 smithi111 ceph-mon[102565]: pgmap v149: 129 pgs: 129 active+clean; 451 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 1.03k op/s 2023-12-12T10:03:30.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:30 smithi173 ceph-mon[103308]: pgmap v149: 129 pgs: 129 active+clean; 451 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 1.03k op/s 2023-12-12T10:03:30.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:30 smithi042 ceph-mon[98894]: pgmap v149: 129 pgs: 129 active+clean; 451 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.3 MiB/s wr, 1.03k op/s 2023-12-12T10:03:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.569+0000 7f6b91ed0700 1 -- 172.21.15.42:0/236289031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 msgr2=0x7f6b8c10d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.569+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/236289031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 0x7f6b8c10d220 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400b0a0 tx=0x7f6b8400b3b0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.569+0000 7f6b91ed0700 1 -- 172.21.15.42:0/236289031 shutdown_connections 2023-12-12T10:03:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.569+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/236289031 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 0x7f6b8c10d220 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.569+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/236289031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c106480 0x7f6b8c106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.569+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/236289031 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b8c1003d0 0x7f6b8c100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.569+0000 7f6b91ed0700 1 -- 172.21.15.42:0/236289031 >> 172.21.15.42:0/236289031 conn(0x7f6b8c077570 msgr2=0x7f6b8c077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.570+0000 7f6b91ed0700 1 -- 172.21.15.42:0/236289031 shutdown_connections 2023-12-12T10:03:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.570+0000 7f6b91ed0700 1 -- 172.21.15.42:0/236289031 wait complete. 2023-12-12T10:03:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.570+0000 7f6b91ed0700 1 Processor -- start 2023-12-12T10:03:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.570+0000 7f6b91ed0700 1 -- start start 2023-12-12T10:03:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b91ed0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c1003d0 0x7f6b8c19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b91ed0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 0x7f6b8c19dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b91ed0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b8c106480 0x7f6b8c197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b91ed0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c197eb0 con 0x7f6b8c100d90 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b91ed0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c198020 con 0x7f6b8c106480 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b91ed0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8c1981e0 con 0x7f6b8c1003d0 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 0x7f6b8c19dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c1003d0 0x7f6b8c19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 0x7f6b8c19dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36694/0 (socket says 172.21.15.42:36694) 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8affd700 1 -- 172.21.15.42:0/2600890341 learned_addr learned my addr 172.21.15.42:0/2600890341 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b8c106480 0x7f6b8c197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/2600890341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b8c106480 msgr2=0x7f6b8c197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b8c106480 0x7f6b8c197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 msgr2=0x7f6b8c19dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 0x7f6b8c19dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/2600890341 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b84009d00 con 0x7f6b8c1003d0 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8affd700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 0x7f6b8c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:03:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8bfff700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b8c106480 0x7f6b8c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:03:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.571+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c1003d0 0x7f6b8c19d6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00e9f0 tx=0x7f6b7c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.572+0000 7f6b88ff9700 1 -- 172.21.15.42:0/2600890341 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c00c9c0 con 0x7f6b8c1003d0 2023-12-12T10:03:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.572+0000 7f6b88ff9700 1 -- 172.21.15.42:0/2600890341 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b7c010430 con 0x7f6b8c1003d0 2023-12-12T10:03:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.572+0000 7f6b91ed0700 1 -- 172.21.15.42:0/2600890341 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8c1984c0 con 0x7f6b8c1003d0 2023-12-12T10:03:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.572+0000 7f6b88ff9700 1 -- 172.21.15.42:0/2600890341 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c018680 con 0x7f6b8c1003d0 2023-12-12T10:03:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.572+0000 7f6b91ed0700 1 -- 172.21.15.42:0/2600890341 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c1a46e0 con 0x7f6b8c1003d0 2023-12-12T10:03:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.574+0000 7f6b88ff9700 1 -- 172.21.15.42:0/2600890341 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f6b7c0187e0 con 0x7f6b8c1003d0 2023-12-12T10:03:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.575+0000 7f6b88ff9700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f6b740a15f0 0x7f6b740a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.575+0000 7f6b8affd700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f6b740a15f0 0x7f6b740a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.575+0000 7f6b88ff9700 1 -- 172.21.15.42:0/2600890341 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6b7c014070 con 0x7f6b8c1003d0 2023-12-12T10:03:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8affd700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f6b740a15f0 0x7f6b740a3ab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c198f30 tx=0x7f6b80009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b9e0c9b80 1 Processor -- start 2023-12-12T10:03:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b9e0c9b80 1 -- start start 2023-12-12T10:03:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 0x5625870b6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562587008af0 con 0x5625870b24a0 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562587006410 con 0x5625870b1b80 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562587001d00 con 0x5625870b62d0 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36702/0 (socket says 172.21.15.42:36702) 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8affd700 1 -- 172.21.15.42:0/3628299287 learned_addr learned my addr 172.21.15.42:0/3628299287 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8bfff700 1 --2- 172.21.15.42:0/3628299287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 0x5625870b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/3628299287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 msgr2=0x5625870b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/3628299287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 0x5625870b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/3628299287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 msgr2=0x5625870b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/3628299287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/3628299287 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562586e7e5d0 con 0x5625870b1b80 2023-12-12T10:03:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.576+0000 7f6b8affd700 1 --2- 172.21.15.42:0/3628299287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:03:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.577+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/3628299287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870b1f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0ca9c0 tx=0x7f6b7c099e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bd917dd83f99985 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.577+0000 7f6b71ffb700 1 -- 172.21.15.42:0/3628299287 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c09a250 con 0x5625870b1b80 2023-12-12T10:03:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.577+0000 7f6b71ffb700 1 -- 172.21.15.42:0/3628299287 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b7c09e3a0 con 0x5625870b1b80 2023-12-12T10:03:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.577+0000 7f6b71ffb700 1 -- 172.21.15.42:0/3628299287 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0109d0 con 0x5625870b1b80 2023-12-12T10:03:32.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3628299287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 msgr2=0x5625870b1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/3628299287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870b1f60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0ca9c0 tx=0x7f6b7c099e10 comp rx=0 tx=0).stop 2023-12-12T10:03:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3628299287 shutdown_connections 2023-12-12T10:03:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/3628299287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870b1f60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/3628299287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 0x5625870b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/3628299287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3628299287 >> 172.21.15.42:0/3628299287 conn(0x562586fdc780 msgr2=0x5625870a7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3628299287 shutdown_connections 2023-12-12T10:03:32.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3628299287 wait complete. 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.578+0000 7f6b9e0c9b80 1 -- start start 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b9e0c9b80 1 Processor -- start 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b1b80 0x5625870bda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b24a0 0x5625870bad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 0x5625870bb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562586fbb500 con 0x5625870b62d0 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562586fbb1a0 con 0x5625870b24a0 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562586e808c0 con 0x5625870b1b80 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b9e0c9b80 10 client.0 _ll_register_callbacks cb 0x562586ee6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:03:32.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b1b80 0x5625870bda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b1b80 0x5625870bda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51690/0 (socket says 172.21.15.42:51690) 2023-12-12T10:03:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8bfff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 0x5625870bb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/1613239213 learned_addr learned my addr 172.21.15.42:0/1613239213 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8affd700 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b24a0 0x5625870bad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/1613239213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b24a0 msgr2=0x5625870bad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b24a0 0x5625870bad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/1613239213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 msgr2=0x5625870bb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:32.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 0x5625870bb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/1613239213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b80017070 con 0x5625870b1b80 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.579+0000 7f6b8bfff700 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 0x5625870bb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b1b80 0x5625870bda90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0cafc0 tx=0x7f6b7c0107d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69d53c33b5737630 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b6affd700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b6affd700 1 -- 172.21.15.42:0/1613239213 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c09a250 con 0x5625870b1b80 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b6affd700 1 -- 172.21.15.42:0/1613239213 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6b7c09e050 con 0x5625870b1b80 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/1613239213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5625870a0730 con 0x5625870b1b80 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b6affd700 1 -- 172.21.15.42:0/1613239213 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b7c0b82c0 con 0x5625870b1b80 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b9e0c9b80 10 client.25576 fetch_fsmap learned FSMap version 37 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/1613239213 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f6b8c1a46e0 con 0x5625870b1b80 2023-12-12T10:03:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b6affd700 1 -- 172.21.15.42:0/1613239213 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b7c07d020 con 0x5625870b1b80 2023-12-12T10:03:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b6affd700 1 -- 172.21.15.42:0/1613239213 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f6b7c099740 con 0x5625870b1b80 2023-12-12T10:03:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b9e0c9b80 10 client.25576 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:03:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b9e0c9b80 10 client.25576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:03:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b9e0c9b80 20 client.25576 populate_metadata read hostname 'smithi042' 2023-12-12T10:03:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625870cb170 0x5625870cd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b9e0c9b80 4 client.25576 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:03:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.580+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/1613239213 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56258707cee0 con 0x5625870cb170 2023-12-12T10:03:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.581+0000 7f6b8affd700 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625870cb170 0x5625870cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.581+0000 7f6b8affd700 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625870cb170 0x5625870cd590 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bff82dc57d2c1db1 server_cookie=f60ea1bf9d038e62 in_seq=0 out_seq=0 2023-12-12T10:03:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.581+0000 7f6b6affd700 10 client.25576 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.586+0000 7f6b6affd700 1 -- 172.21.15.42:0/1613239213 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56258707cee0 con 0x5625870cb170 2023-12-12T10:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.586+0000 7f6b6affd700 10 client.25576 handle_command_reply: tid=0 2023-12-12T10:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.586+0000 7f6b9e0c9b80 1 client.25576 shutdown 2023-12-12T10:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.586+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/1613239213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b1b80 msgr2=0x5625870bda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.586+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b1b80 0x5625870bda90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0cafc0 tx=0x7f6b7c0107d0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/1613239213 shutdown_connections 2023-12-12T10:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 0x5625870bb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b1b80 0x5625870bda90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625870cb170 0x5625870cd590 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/1613239213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b24a0 0x5625870bad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b6affd700 0 client.25576 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/1613239213 >> 172.21.15.42:0/1613239213 conn(0x562586fdc780 msgr2=0x5625870ac570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/1613239213 shutdown_connections 2023-12-12T10:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/1613239213 wait complete. 2023-12-12T10:03:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.587+0000 7f6b9e0c9b80 20 client.25576 trim_cache size 0 max 16384 2023-12-12T10:03:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b9e0c9b80 1 Processor -- start 2023-12-12T10:03:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b9e0c9b80 1 -- start start 2023-12-12T10:03:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870acec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 0x5625870d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625870a9930 con 0x5625870b24a0 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625870b5020 con 0x5625870b1b80 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625870c7260 con 0x5625870b62d0 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870acec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8affd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870acec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39774/0 (socket says 172.21.15.42:39774) 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 0x5625870d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/3780588313 learned_addr learned my addr 172.21.15.42:0/3780588313 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/3780588313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 msgr2=0x5625870d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/3780588313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 0x5625870d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/3780588313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 msgr2=0x5625870a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/3780588313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870a79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.603+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/3780588313 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562586e2ad80 con 0x5625870b1b80 2023-12-12T10:03:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.604+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/3780588313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870acec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c096cc0 tx=0x7f6b7c010a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d647c934ace5d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.604+0000 7f6b727fc700 1 -- 172.21.15.42:0/3780588313 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c09e160 con 0x5625870b1b80 2023-12-12T10:03:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.604+0000 7f6b727fc700 1 -- 172.21.15.42:0/3780588313 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b7c07d030 con 0x5625870b1b80 2023-12-12T10:03:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.604+0000 7f6b727fc700 1 -- 172.21.15.42:0/3780588313 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0b82c0 con 0x5625870b1b80 2023-12-12T10:03:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3780588313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 msgr2=0x5625870acec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/3780588313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870acec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c096cc0 tx=0x7f6b7c010a70 comp rx=0 tx=0).stop 2023-12-12T10:03:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3780588313 shutdown_connections 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/3780588313 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870acec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/3780588313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b62d0 0x5625870d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/3780588313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b24a0 0x5625870a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3780588313 >> 172.21.15.42:0/3780588313 conn(0x562586fdc780 msgr2=0x5625870bd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3780588313 shutdown_connections 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/3780588313 wait complete. 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 -- start start 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.605+0000 7f6b9e0c9b80 1 Processor -- start 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870c03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b24a0 0x5625870c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b9e0c9b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 0x5625870c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562587120790 con 0x5625870b62d0 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562587120900 con 0x5625870b1b80 2023-12-12T10:03:32.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b9e0c9b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562587120ac0 con 0x5625870b24a0 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b9e0c9b80 10 client.0 _ll_register_callbacks cb 0x562586ee6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870c03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870c03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39780/0 (socket says 172.21.15.42:39780) 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/518524905 learned_addr learned my addr 172.21.15.42:0/518524905 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8bfff700 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 0x5625870c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8affd700 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b24a0 0x5625870c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/518524905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b24a0 msgr2=0x5625870c0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b24a0 0x5625870c0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/518524905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 msgr2=0x5625870c0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 0x5625870c0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8b7fe700 1 -- 172.21.15.42:0/518524905 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8c1a46e0 con 0x5625870b1b80 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b8b7fe700 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870c03d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0a20b0 tx=0x7f6b7c098800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb6097ad978db649 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.606+0000 7f6b6affd700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:03:32.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b6affd700 1 -- 172.21.15.42:0/518524905 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c09a250 con 0x5625870b1b80 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b6affd700 1 -- 172.21.15.42:0/518524905 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6b7c07e080 con 0x5625870b1b80 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/518524905 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b7c09e3a0 con 0x5625870b1b80 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b6affd700 1 -- 172.21.15.42:0/518524905 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b7c0a23c0 con 0x5625870b1b80 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b9e0c9b80 10 client.25640 fetch_fsmap learned FSMap version 37 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/518524905 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562586e13c00 con 0x5625870b1b80 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b6affd700 1 -- 172.21.15.42:0/518524905 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b7c09e3a0 con 0x5625870b1b80 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b6affd700 1 -- 172.21.15.42:0/518524905 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f6b7c096160 con 0x5625870b1b80 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b9e0c9b80 10 client.25640 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b9e0c9b80 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b9e0c9b80 20 client.25640 populate_metadata read hostname 'smithi042' 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625870ad910 0x5625870afd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:32.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b9e0c9b80 4 client.25640 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:03:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/518524905 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562586fc7f70 con 0x5625870ad910 2023-12-12T10:03:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.607+0000 7f6b8affd700 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625870ad910 0x5625870afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.608+0000 7f6b6affd700 10 client.25640 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:32.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:32.608+0000 7f6b8affd700 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625870ad910 0x5625870afd30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a2ee2e34fc056f server_cookie=a4d68a7ceee4b568 in_seq=0 out_seq=0 2023-12-12T10:03:32.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:32 smithi111 ceph-mon[102565]: pgmap v150: 129 pgs: 129 active+clean; 451 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 714 op/s 2023-12-12T10:03:32.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:32 smithi173 ceph-mon[103308]: pgmap v150: 129 pgs: 129 active+clean; 451 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 714 op/s 2023-12-12T10:03:32.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:32 smithi042 ceph-mon[98894]: pgmap v150: 129 pgs: 129 active+clean; 451 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.6 MiB/s wr, 714 op/s 2023-12-12T10:03:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.443+0000 7f6b6affd700 1 -- 172.21.15.42:0/518524905 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562586fc7f70 con 0x5625870ad910 2023-12-12T10:03:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.443+0000 7f6b6affd700 10 client.25640 handle_command_reply: tid=0 2023-12-12T10:03:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.443+0000 7f6b9e0c9b80 1 client.25640 shutdown 2023-12-12T10:03:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.445+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/518524905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 msgr2=0x5625870c03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.445+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870c03d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c0a20b0 tx=0x7f6b7c098800 comp rx=0 tx=0).stop 2023-12-12T10:03:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.446+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/518524905 shutdown_connections 2023-12-12T10:03:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.446+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5625870ad910 0x5625870afd30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.446+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5625870b1b80 0x5625870c03d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.446+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5625870b62d0 0x5625870c0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.446+0000 7f6b9e0c9b80 1 --2- 172.21.15.42:0/518524905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5625870b24a0 0x5625870c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.446+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/518524905 >> 172.21.15.42:0/518524905 conn(0x562586fdc780 msgr2=0x5625870b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.451+0000 7f6b6affd700 0 client.25640 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.452+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/518524905 shutdown_connections 2023-12-12T10:03:34.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.452+0000 7f6b9e0c9b80 1 -- 172.21.15.42:0/518524905 wait complete. 2023-12-12T10:03:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.455+0000 7f6b9e0c9b80 20 client.25640 trim_cache size 0 max 16384 2023-12-12T10:03:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:03:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:03:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:03:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:03:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.456+0000 7f6b91ed0700 1 -- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f6b740a15f0 msgr2=0x7f6b740a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.456+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f6b740a15f0 0x7f6b740a3ab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c198f30 tx=0x7f6b80009260 comp rx=0 tx=0).stop 2023-12-12T10:03:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.456+0000 7f6b91ed0700 1 -- 172.21.15.42:0/2600890341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c1003d0 msgr2=0x7f6b8c19d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.456+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c1003d0 0x7f6b8c19d6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00e9f0 tx=0x7f6b7c00ed00 comp rx=0 tx=0).stop 2023-12-12T10:03:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.461+0000 7f6b91ed0700 1 -- 172.21.15.42:0/2600890341 shutdown_connections 2023-12-12T10:03:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.461+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f6b740a15f0 0x7f6b740a3ab0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.461+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6b8c106480 0x7f6b8c197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.461+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b8c1003d0 0x7f6b8c19d6b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.461+0000 7f6b91ed0700 1 --2- 172.21.15.42:0/2600890341 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b8c100d90 0x7f6b8c19dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:34.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.461+0000 7f6b91ed0700 1 -- 172.21.15.42:0/2600890341 >> 172.21.15.42:0/2600890341 conn(0x7f6b8c077570 msgr2=0x7f6b8c068ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:34.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.466+0000 7f6b91ed0700 1 -- 172.21.15.42:0/2600890341 shutdown_connections 2023-12-12T10:03:34.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:34.467+0000 7f6b91ed0700 1 -- 172.21.15.42:0/2600890341 wait complete. 2023-12-12T10:03:34.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:34 smithi111 ceph-mon[102565]: pgmap v151: 129 pgs: 129 active+clean; 453 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 731 op/s 2023-12-12T10:03:34.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:34 smithi173 ceph-mon[103308]: pgmap v151: 129 pgs: 129 active+clean; 453 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 731 op/s 2023-12-12T10:03:34.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:34 smithi042 ceph-mon[98894]: pgmap v151: 129 pgs: 129 active+clean; 453 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.7 MiB/s wr, 731 op/s 2023-12-12T10:03:36.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:36 smithi111 ceph-mon[102565]: pgmap v152: 129 pgs: 129 active+clean; 467 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 945 op/s 2023-12-12T10:03:36.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:36 smithi173 ceph-mon[103308]: pgmap v152: 129 pgs: 129 active+clean; 467 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 945 op/s 2023-12-12T10:03:36.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:36 smithi042 ceph-mon[98894]: pgmap v152: 129 pgs: 129 active+clean; 467 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 5.7 MiB/s wr, 945 op/s 2023-12-12T10:03:37.823 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:03:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-i[140500]: 2023-12-12T10:03:37.535+0000 7fd22c59b700 -1 mds.1.bal find_exports balancer runs too long 2023-12-12T10:03:38.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:38 smithi111 ceph-mon[102565]: pgmap v153: 129 pgs: 129 active+clean; 467 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2023-12-12T10:03:38.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:38 smithi173 ceph-mon[103308]: pgmap v153: 129 pgs: 129 active+clean; 467 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2023-12-12T10:03:38.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:38 smithi042 ceph-mon[98894]: pgmap v153: 129 pgs: 129 active+clean; 467 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.9 MiB/s wr, 538 op/s 2023-12-12T10:03:40.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:40 smithi111 ceph-mon[102565]: pgmap v154: 129 pgs: 129 active+clean; 490 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 875 op/s 2023-12-12T10:03:40.829 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:40 smithi173 ceph-mon[103308]: pgmap v154: 129 pgs: 129 active+clean; 490 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 875 op/s 2023-12-12T10:03:40.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:40 smithi042 ceph-mon[98894]: pgmap v154: 129 pgs: 129 active+clean; 490 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.6 MiB/s wr, 875 op/s 2023-12-12T10:03:42.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:42 smithi111 ceph-mon[102565]: pgmap v155: 129 pgs: 129 active+clean; 490 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 603 op/s 2023-12-12T10:03:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:42 smithi173 ceph-mon[103308]: pgmap v155: 129 pgs: 129 active+clean; 490 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 603 op/s 2023-12-12T10:03:42.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:42 smithi042 ceph-mon[98894]: pgmap v155: 129 pgs: 129 active+clean; 490 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.0 MiB/s wr, 603 op/s 2023-12-12T10:03:44.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:44 smithi111 ceph-mon[102565]: pgmap v156: 129 pgs: 129 active+clean; 491 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 613 op/s 2023-12-12T10:03:44.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:44 smithi173 ceph-mon[103308]: pgmap v156: 129 pgs: 129 active+clean; 491 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 613 op/s 2023-12-12T10:03:44.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:44 smithi042 ceph-mon[98894]: pgmap v156: 129 pgs: 129 active+clean; 491 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.1 MiB/s wr, 613 op/s 2023-12-12T10:03:46.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:46 smithi111 ceph-mon[102565]: pgmap v157: 129 pgs: 129 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 754 op/s 2023-12-12T10:03:46.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:46 smithi173 ceph-mon[103308]: pgmap v157: 129 pgs: 129 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 754 op/s 2023-12-12T10:03:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:46 smithi042 ceph-mon[98894]: pgmap v157: 129 pgs: 129 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.5 MiB/s wr, 754 op/s 2023-12-12T10:03:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.653+0000 7fd829b62700 1 -- 172.21.15.42:0/1793415573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd824108ef0 msgr2=0x7fd8241141e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:47.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.653+0000 7fd829b62700 1 --2- 172.21.15.42:0/1793415573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd824108ef0 0x7fd8241141e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0099c0 tx=0x7fd80c009cd0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.656+0000 7fd829b62700 1 -- 172.21.15.42:0/1793415573 shutdown_connections 2023-12-12T10:03:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.656+0000 7fd829b62700 1 --2- 172.21.15.42:0/1793415573 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8241085d0 0x7fd8241089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.656+0000 7fd829b62700 1 --2- 172.21.15.42:0/1793415573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824114720 0x7fd824116b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.656+0000 7fd829b62700 1 --2- 172.21.15.42:0/1793415573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd824108ef0 0x7fd8241141e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.656+0000 7fd829b62700 1 -- 172.21.15.42:0/1793415573 >> 172.21.15.42:0/1793415573 conn(0x7fd82406c5a0 msgr2=0x7fd82406c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.658+0000 7fd829b62700 1 -- 172.21.15.42:0/1793415573 shutdown_connections 2023-12-12T10:03:47.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.658+0000 7fd829b62700 1 -- 172.21.15.42:0/1793415573 wait complete. 2023-12-12T10:03:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.659+0000 7fd829b62700 1 Processor -- start 2023-12-12T10:03:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.659+0000 7fd829b62700 1 -- start start 2023-12-12T10:03:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd829b62700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8241085d0 0x7fd8241ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd829b62700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824108ef0 0x7fd8241acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd829b62700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd824114720 0x7fd824110d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd829b62700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8241114f0 con 0x7fd824114720 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd829b62700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd824111660 con 0x7fd8241085d0 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd829b62700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd824111820 con 0x7fd824108ef0 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd8237fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8241085d0 0x7fd8241ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd823fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd824114720 0x7fd824110d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd81affd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824108ef0 0x7fd8241acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd823fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd824114720 0x7fd824110d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36738/0 (socket says 172.21.15.42:36738) 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd823fff700 1 -- 172.21.15.42:0/2909935891 learned_addr learned my addr 172.21.15.42:0/2909935891 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd8237fe700 1 -- 172.21.15.42:0/2909935891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824108ef0 msgr2=0x7fd8241acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd8237fe700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824108ef0 0x7fd8241acad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd8237fe700 1 -- 172.21.15.42:0/2909935891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd824114720 msgr2=0x7fd824110d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd8237fe700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd824114720 0x7fd824110d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.660+0000 7fd8237fe700 1 -- 172.21.15.42:0/2909935891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c0095f0 con 0x7fd8241085d0 2023-12-12T10:03:47.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.661+0000 7fd8237fe700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8241085d0 0x7fd8241ac590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd81400e9f0 tx=0x7fd81400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.664+0000 7fd8217fa700 1 -- 172.21.15.42:0/2909935891 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400eee0 con 0x7fd8241085d0 2023-12-12T10:03:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.664+0000 7fd8217fa700 1 -- 172.21.15.42:0/2909935891 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81400cbd0 con 0x7fd8241085d0 2023-12-12T10:03:47.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.664+0000 7fd829b62700 1 -- 172.21.15.42:0/2909935891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd824111b00 con 0x7fd8241085d0 2023-12-12T10:03:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.664+0000 7fd8217fa700 1 -- 172.21.15.42:0/2909935891 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814018680 con 0x7fd8241085d0 2023-12-12T10:03:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.664+0000 7fd829b62700 1 -- 172.21.15.42:0/2909935891 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8241b34f0 con 0x7fd8241085d0 2023-12-12T10:03:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.669+0000 7fd8217fa700 1 -- 172.21.15.42:0/2909935891 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fd814010430 con 0x7fd8241085d0 2023-12-12T10:03:47.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.669+0000 7fd835d5bb80 1 Processor -- start 2023-12-12T10:03:47.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd835d5bb80 1 -- start start 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76bd40 0x5629cd76c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 0x5629cd766580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd766ac0 0x5629cd766f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd6c1af0 con 0x5629cd76bd40 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd6bf410 con 0x5629cd76c660 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd6bad00 con 0x5629cd766ac0 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd8217fa700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd8100a1670 0x7fd8100a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd8237fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 0x5629cd766580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd8237fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 0x5629cd766580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39802/0 (socket says 172.21.15.42:39802) 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd8237fe700 1 -- 172.21.15.42:0/3250384363 learned_addr learned my addr 172.21.15.42:0/3250384363 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd8217fa700 1 -- 172.21.15.42:0/2909935891 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd814014070 con 0x7fd8241085d0 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.670+0000 7fd823fff700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd8100a1670 0x7fd8100a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd8237fe700 1 -- 172.21.15.42:0/3250384363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd766ac0 msgr2=0x5629cd766f00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd8237fe700 1 --2- 172.21.15.42:0/3250384363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd766ac0 0x5629cd766f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd8237fe700 1 -- 172.21.15.42:0/3250384363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76bd40 msgr2=0x5629cd76c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd8237fe700 1 --2- 172.21.15.42:0/3250384363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76bd40 0x5629cd76c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd8237fe700 1 -- 172.21.15.42:0/3250384363 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629cd5375d0 con 0x5629cd76c660 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd8237fe700 1 --2- 172.21.15.42:0/3250384363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 0x5629cd766580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd81400e9c0 tx=0x7fd81409a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea4854ac3c40402b server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd823fff700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd8100a1670 0x7fd8100a3b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009f60 tx=0x7fd81c009920 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd819ffb700 1 -- 172.21.15.42:0/3250384363 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81409aa40 con 0x5629cd76c660 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd819ffb700 1 -- 172.21.15.42:0/3250384363 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81409a340 con 0x5629cd76c660 2023-12-12T10:03:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.671+0000 7fd819ffb700 1 -- 172.21.15.42:0/3250384363 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81409a5a0 con 0x5629cd76c660 2023-12-12T10:03:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.672+0000 7fd835d5bb80 1 -- 172.21.15.42:0/3250384363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 msgr2=0x5629cd766580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.672+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/3250384363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 0x5629cd766580 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd81400e9c0 tx=0x7fd81409a110 comp rx=0 tx=0).stop 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.672+0000 7fd835d5bb80 1 -- 172.21.15.42:0/3250384363 shutdown_connections 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.672+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/3250384363 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd766ac0 0x5629cd766f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.672+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/3250384363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76bd40 0x5629cd76c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.672+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/3250384363 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 0x5629cd766580 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.672+0000 7fd835d5bb80 1 -- 172.21.15.42:0/3250384363 >> 172.21.15.42:0/3250384363 conn(0x5629cd695780 msgr2=0x5629cd760c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.672+0000 7fd835d5bb80 1 -- 172.21.15.42:0/3250384363 shutdown_connections 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.672+0000 7fd835d5bb80 1 -- 172.21.15.42:0/3250384363 wait complete. 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.673+0000 7fd835d5bb80 1 -- start start 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.673+0000 7fd835d5bb80 1 Processor -- start 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd766ac0 0x5629cd76fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd76ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 0x5629cd7704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd674500 con 0x5629cd766ac0 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd6741a0 con 0x5629cd76c660 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd5398c0 con 0x5629cd76bd40 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd835d5bb80 10 client.0 _ll_register_callbacks cb 0x5629cd59fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd81affd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd76ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd81affd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd76ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51734/0 (socket says 172.21.15.42:51734) 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd81affd700 1 -- 172.21.15.42:0/609538167 learned_addr learned my addr 172.21.15.42:0/609538167 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd81affd700 1 -- 172.21.15.42:0/609538167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 msgr2=0x5629cd7704c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd81affd700 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 0x5629cd7704c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd81affd700 1 -- 172.21.15.42:0/609538167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd766ac0 msgr2=0x5629cd76fa40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd81affd700 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd766ac0 0x5629cd76fa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.674+0000 7fd81affd700 1 -- 172.21.15.42:0/609538167 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8241b34f0 con 0x5629cd76bd40 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.676+0000 7fd81affd700 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd76ff80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0058e0 tx=0x7fd80c005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=937dd8c4884f9922 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.676+0000 7fd802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd802ffd700 1 -- 172.21.15.42:0/609538167 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c00bba0 con 0x5629cd76bd40 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd802ffd700 1 -- 172.21.15.42:0/609538167 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd80c00bd00 con 0x5629cd76bd40 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd835d5bb80 1 -- 172.21.15.42:0/609538167 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629cd759730 con 0x5629cd76bd40 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd802ffd700 1 -- 172.21.15.42:0/609538167 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd80c00bec0 con 0x5629cd76bd40 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd835d5bb80 10 client.25591 fetch_fsmap learned FSMap version 37 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd835d5bb80 1 -- 172.21.15.42:0/609538167 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5629cd4e3d80 con 0x5629cd76bd40 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd802ffd700 1 -- 172.21.15.42:0/609538167 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5629cd759730 con 0x5629cd76bd40 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd802ffd700 1 -- 172.21.15.42:0/609538167 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fd80c00f840 con 0x5629cd76bd40 2023-12-12T10:03:47.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd835d5bb80 10 client.25591 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:03:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd835d5bb80 10 client.25591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:03:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.677+0000 7fd835d5bb80 20 client.25591 populate_metadata read hostname 'smithi042' 2023-12-12T10:03:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.678+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5629cd780200 0x5629cd782620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.678+0000 7fd835d5bb80 4 client.25591 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:03:47.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.678+0000 7fd835d5bb80 1 -- 172.21.15.42:0/609538167 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629cd735ee0 con 0x5629cd780200 2023-12-12T10:03:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.678+0000 7fd8237fe700 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5629cd780200 0x5629cd782620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.678+0000 7fd802ffd700 10 client.25591 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.678+0000 7fd8237fe700 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5629cd780200 0x5629cd782620 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4610d131dbea1b46 server_cookie=30778c892f796f9d in_seq=0 out_seq=0 2023-12-12T10:03:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd802ffd700 1 -- 172.21.15.42:0/609538167 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629cd735ee0 con 0x5629cd780200 2023-12-12T10:03:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd802ffd700 10 client.25591 handle_command_reply: tid=0 2023-12-12T10:03:47.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 client.25591 shutdown 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 -- 172.21.15.42:0/609538167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 msgr2=0x5629cd76ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd76ff80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0058e0 tx=0x7fd80c005770 comp rx=0 tx=0).stop 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 -- 172.21.15.42:0/609538167 shutdown_connections 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5629cd780200 0x5629cd782620 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd76c660 0x5629cd7704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd802ffd700 0 client.25591 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd766ac0 0x5629cd76fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/609538167 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd76ff80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 -- 172.21.15.42:0/609538167 >> 172.21.15.42:0/609538167 conn(0x5629cd695780 msgr2=0x5629cd765570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 -- 172.21.15.42:0/609538167 shutdown_connections 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.679+0000 7fd835d5bb80 1 -- 172.21.15.42:0/609538167 wait complete. 2023-12-12T10:03:47.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.680+0000 7fd835d5bb80 20 client.25591 trim_cache size 0 max 16384 2023-12-12T10:03:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd835d5bb80 1 Processor -- start 2023-12-12T10:03:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd835d5bb80 1 -- start start 2023-12-12T10:03:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 0x5629cd77bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd760c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd769f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd761d70 con 0x5629cd76c660 2023-12-12T10:03:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd77d720 con 0x5629cd766ac0 2023-12-12T10:03:47.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd761f60 con 0x5629cd76bd40 2023-12-12T10:03:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd823fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd769f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd823fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd769f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36758/0 (socket says 172.21.15.42:36758) 2023-12-12T10:03:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd823fff700 1 -- 172.21.15.42:0/667007666 learned_addr learned my addr 172.21.15.42:0/667007666 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd81affd700 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd760c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd8237fe700 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 0x5629cd77bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd81affd700 1 -- 172.21.15.42:0/667007666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 msgr2=0x5629cd77bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd81affd700 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 0x5629cd77bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd81affd700 1 -- 172.21.15.42:0/667007666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 msgr2=0x5629cd769f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd81affd700 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd769f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd81affd700 1 -- 172.21.15.42:0/667007666 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629cd4ccc00 con 0x5629cd76bd40 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.696+0000 7fd823fff700 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd769f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.697+0000 7fd81affd700 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd760c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd80c021000 tx=0x7fd80c005b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e719a97eefe630 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.697+0000 7fd81a7fc700 1 -- 172.21.15.42:0/667007666 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c00bba0 con 0x5629cd76bd40 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.697+0000 7fd81a7fc700 1 -- 172.21.15.42:0/667007666 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd80c01e040 con 0x5629cd76bd40 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.697+0000 7fd81a7fc700 1 -- 172.21.15.42:0/667007666 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0039f0 con 0x5629cd76bd40 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 -- 172.21.15.42:0/667007666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 msgr2=0x5629cd760c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd760c20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd80c021000 tx=0x7fd80c005b00 comp rx=0 tx=0).stop 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 -- 172.21.15.42:0/667007666 shutdown_connections 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 0x5629cd77bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd769f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/667007666 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd760c20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 -- 172.21.15.42:0/667007666 >> 172.21.15.42:0/667007666 conn(0x5629cd695780 msgr2=0x5629cd76f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 -- 172.21.15.42:0/667007666 shutdown_connections 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 -- 172.21.15.42:0/667007666 wait complete. 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 -- start start 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.698+0000 7fd835d5bb80 1 Processor -- start 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 0x5629cd774fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd775510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd835d5bb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd775a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd7d5880 con 0x5629cd76c660 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd7d59f0 con 0x5629cd766ac0 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd835d5bb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629cd7d5bb0 con 0x5629cd76bd40 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd835d5bb80 10 client.0 _ll_register_callbacks cb 0x5629cd59fa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd823fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd775a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd8237fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 0x5629cd774fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd81affd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd775510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd823fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd775a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36762/0 (socket says 172.21.15.42:36762) 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd823fff700 1 -- 172.21.15.42:0/4001355753 learned_addr learned my addr 172.21.15.42:0/4001355753 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd823fff700 1 -- 172.21.15.42:0/4001355753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 msgr2=0x5629cd775510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd823fff700 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd775510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd823fff700 1 -- 172.21.15.42:0/4001355753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 msgr2=0x5629cd774fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd823fff700 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 0x5629cd774fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd823fff700 1 -- 172.21.15.42:0/4001355753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c008e00 con 0x5629cd76c660 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd823fff700 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd775a50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0179e0 tx=0x7fd81c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48e15ccc63daf30 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.699+0000 7fd802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd802ffd700 1 -- 172.21.15.42:0/4001355753 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c01c040 con 0x5629cd76c660 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd835d5bb80 1 -- 172.21.15.42:0/4001355753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd81409a340 con 0x5629cd76c660 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd802ffd700 1 -- 172.21.15.42:0/4001355753 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd81c004bb0 con 0x5629cd76c660 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd802ffd700 1 -- 172.21.15.42:0/4001355753 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd81c00e070 con 0x5629cd76c660 2023-12-12T10:03:47.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd835d5bb80 10 client.15969 fetch_fsmap learned FSMap version 37 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd835d5bb80 1 -- 172.21.15.42:0/4001355753 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x5629cd4e3d80 con 0x5629cd76c660 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd802ffd700 1 -- 172.21.15.42:0/4001355753 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd81409a340 con 0x5629cd76c660 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd802ffd700 1 -- 172.21.15.42:0/4001355753 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fd81c023710 con 0x5629cd76c660 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd835d5bb80 10 client.15969 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd835d5bb80 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd835d5bb80 20 client.15969 populate_metadata read hostname 'smithi042' 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5629cd7d7520 0x5629cd7d9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd835d5bb80 4 client.15969 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd835d5bb80 1 -- 172.21.15.42:0/4001355753 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5629cd680f70 con 0x5629cd7d7520 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.700+0000 7fd8237fe700 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5629cd7d7520 0x5629cd7d9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.701+0000 7fd802ffd700 10 client.15969 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:47.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.701+0000 7fd8237fe700 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5629cd7d7520 0x5629cd7d9940 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46fe9912cdee0644 server_cookie=43080ebcffe044fd in_seq=0 out_seq=0 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.828+0000 7fd802ffd700 1 -- 172.21.15.42:0/4001355753 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5629cd680f70 con 0x5629cd7d7520 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.828+0000 7fd802ffd700 10 client.15969 handle_command_reply: tid=0 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.828+0000 7fd835d5bb80 1 client.15969 shutdown 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 -- 172.21.15.42:0/4001355753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 msgr2=0x5629cd775a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd775a50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd81c0179e0 tx=0x7fd81c004b40 comp rx=0 tx=0).stop 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 -- 172.21.15.42:0/4001355753 shutdown_connections 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x5629cd7d7520 0x5629cd7d9940 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x5629cd766ac0 0x5629cd774fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5629cd76c660 0x5629cd775a50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd802ffd700 0 client.15969 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 --2- 172.21.15.42:0/4001355753 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x5629cd76bd40 0x5629cd775510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 -- 172.21.15.42:0/4001355753 >> 172.21.15.42:0/4001355753 conn(0x5629cd695780 msgr2=0x5629cd760b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 -- 172.21.15.42:0/4001355753 shutdown_connections 2023-12-12T10:03:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 1 -- 172.21.15.42:0/4001355753 wait complete. 2023-12-12T10:03:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.829+0000 7fd835d5bb80 20 client.15969 trim_cache size 0 max 16384 2023-12-12T10:03:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:03:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:03:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:03:47.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 -- 172.21.15.42:0/2909935891 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd8100a1670 msgr2=0x7fd8100a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd8100a1670 0x7fd8100a3b30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd81c009f60 tx=0x7fd81c009920 comp rx=0 tx=0).stop 2023-12-12T10:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 -- 172.21.15.42:0/2909935891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8241085d0 msgr2=0x7fd8241ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8241085d0 0x7fd8241ac590 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd81400e9f0 tx=0x7fd81400ed00 comp rx=0 tx=0).stop 2023-12-12T10:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 -- 172.21.15.42:0/2909935891 shutdown_connections 2023-12-12T10:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd8100a1670 0x7fd8100a3b30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8241085d0 0x7fd8241ac590 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd824114720 0x7fd824110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 --2- 172.21.15.42:0/2909935891 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd824108ef0 0x7fd8241acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:03:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 -- 172.21.15.42:0/2909935891 >> 172.21.15.42:0/2909935891 conn(0x7fd82406c5a0 msgr2=0x7fd82406ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:03:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 -- 172.21.15.42:0/2909935891 shutdown_connections 2023-12-12T10:03:47.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:03:47.830+0000 7fd829b62700 1 -- 172.21.15.42:0/2909935891 wait complete. 2023-12-12T10:03:48.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:48 smithi111 ceph-mon[102565]: pgmap v158: 129 pgs: 129 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 505 op/s 2023-12-12T10:03:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:48 smithi173 ceph-mon[103308]: pgmap v158: 129 pgs: 129 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 505 op/s 2023-12-12T10:03:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:48 smithi042 ceph-mon[98894]: pgmap v158: 129 pgs: 129 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.2 MiB/s wr, 505 op/s 2023-12-12T10:03:50.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:50 smithi111 ceph-mon[102565]: pgmap v159: 129 pgs: 129 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.5 MiB/s wr, 797 op/s 2023-12-12T10:03:50.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:50 smithi173 ceph-mon[103308]: pgmap v159: 129 pgs: 129 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.5 MiB/s wr, 797 op/s 2023-12-12T10:03:50.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:50 smithi042 ceph-mon[98894]: pgmap v159: 129 pgs: 129 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 5.5 MiB/s wr, 797 op/s 2023-12-12T10:03:52.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:52 smithi111 ceph-mon[102565]: pgmap v160: 129 pgs: 129 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 461 op/s 2023-12-12T10:03:52.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:52 smithi173 ceph-mon[103308]: pgmap v160: 129 pgs: 129 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 461 op/s 2023-12-12T10:03:52.826 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:03:52 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-i[140500]: 2023-12-12T10:03:52.521+0000 7fd22c59b700 -1 mds.1.bal find_exports balancer runs too long 2023-12-12T10:03:52.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:52 smithi042 ceph-mon[98894]: pgmap v160: 129 pgs: 129 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 461 op/s 2023-12-12T10:03:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:54 smithi173 ceph-mon[103308]: pgmap v161: 129 pgs: 129 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 472 op/s 2023-12-12T10:03:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:54 smithi042 ceph-mon[98894]: pgmap v161: 129 pgs: 129 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 472 op/s 2023-12-12T10:03:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:54 smithi111 ceph-mon[102565]: pgmap v161: 129 pgs: 129 active+clean; 519 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.8 MiB/s wr, 472 op/s 2023-12-12T10:03:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:56 smithi173 ceph-mon[103308]: pgmap v162: 129 pgs: 129 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 698 op/s 2023-12-12T10:03:56.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:56 smithi042 ceph-mon[98894]: pgmap v162: 129 pgs: 129 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 698 op/s 2023-12-12T10:03:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:56 smithi111 ceph-mon[102565]: pgmap v162: 129 pgs: 129 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.5 MiB/s wr, 698 op/s 2023-12-12T10:03:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:58 smithi173 ceph-mon[103308]: pgmap v163: 129 pgs: 129 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 540 op/s 2023-12-12T10:03:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:58 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:03:58.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:58 smithi042 ceph-mon[98894]: pgmap v163: 129 pgs: 129 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 540 op/s 2023-12-12T10:03:58.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:58 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:03:58.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:58 smithi111 ceph-mon[102565]: pgmap v163: 129 pgs: 129 active+clean; 540 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.0 MiB/s wr, 540 op/s 2023-12-12T10:03:58.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:58 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:03:59.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:59 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:03:59.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:59 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:03:59.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:03:59 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:03:59.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:59 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:03:59.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:59 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:03:59.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:03:59 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:03:59.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:59 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:03:59.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:59 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:03:59.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:03:59 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:04:00.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:00 smithi173 ceph-mon[103308]: pgmap v164: 129 pgs: 129 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 732 op/s 2023-12-12T10:04:00.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:00 smithi042 ceph-mon[98894]: pgmap v164: 129 pgs: 129 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 732 op/s 2023-12-12T10:04:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.943+0000 7f69861a6700 1 -- 172.21.15.42:0/3730315281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 msgr2=0x7f698010d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.943+0000 7f69861a6700 1 --2- 172.21.15.42:0/3730315281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 0x7f698010d2a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6978009f90 tx=0x7f697800b330 comp rx=0 tx=0).stop 2023-12-12T10:04:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.943+0000 7f69861a6700 1 -- 172.21.15.42:0/3730315281 shutdown_connections 2023-12-12T10:04:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.943+0000 7f69861a6700 1 --2- 172.21.15.42:0/3730315281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 0x7f698010d2a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.943+0000 7f69861a6700 1 --2- 172.21.15.42:0/3730315281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6980106470 0x7f6980106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.943+0000 7f69861a6700 1 --2- 172.21.15.42:0/3730315281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69801003d0 0x7f6980100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.943+0000 7f69861a6700 1 -- 172.21.15.42:0/3730315281 >> 172.21.15.42:0/3730315281 conn(0x7f69800fbea0 msgr2=0x7f69800fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.943+0000 7f69861a6700 1 -- 172.21.15.42:0/3730315281 shutdown_connections 2023-12-12T10:04:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.943+0000 7f69861a6700 1 -- 172.21.15.42:0/3730315281 wait complete. 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f69861a6700 1 Processor -- start 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f69861a6700 1 -- start start 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f69861a6700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69801003d0 0x7f69801986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f69861a6700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 0x7f6980198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f69861a6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6980106470 0x7f69801a3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f69861a6700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6980199420 con 0x7f6980100d90 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f69861a6700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69801a3c70 con 0x7f69801003d0 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f69861a6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69801a3db0 con 0x7f6980106470 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f697effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 0x7f6980198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f697effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 0x7f6980198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36776/0 (socket says 172.21.15.42:36776) 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f697ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6980106470 0x7f69801a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f697effd700 1 -- 172.21.15.42:0/3462912144 learned_addr learned my addr 172.21.15.42:0/3462912144 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.944+0000 7f697f7fe700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69801003d0 0x7f69801986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697effd700 1 -- 172.21.15.42:0/3462912144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6980106470 msgr2=0x7f69801a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697effd700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6980106470 0x7f69801a3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697effd700 1 -- 172.21.15.42:0/3462912144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69801003d0 msgr2=0x7f69801986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697effd700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69801003d0 0x7f69801986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697effd700 1 -- 172.21.15.42:0/3462912144 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6978009c40 con 0x7f6980100d90 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697ffff700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6980106470 0x7f69801a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697effd700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 0x7f6980198bf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9a0 tx=0x7f697000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697cff9700 1 -- 172.21.15.42:0/3462912144 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970009790 con 0x7f6980100d90 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f69861a6700 1 -- 172.21.15.42:0/3462912144 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69801a3fb0 con 0x7f6980100d90 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697cff9700 1 -- 172.21.15.42:0/3462912144 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697000ce80 con 0x7f6980100d90 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f697cff9700 1 -- 172.21.15.42:0/3462912144 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970018680 con 0x7f6980100d90 2023-12-12T10:04:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.945+0000 7f69861a6700 1 -- 172.21.15.42:0/3462912144 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69801a44e0 con 0x7f6980100d90 2023-12-12T10:04:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.948+0000 7f697cff9700 1 -- 172.21.15.42:0/3462912144 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f6970010430 con 0x7f6980100d90 2023-12-12T10:04:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.948+0000 7f697cff9700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f69680a15f0 0x7f69680a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f697cff9700 1 -- 172.21.15.42:0/3462912144 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6970014070 con 0x7f6980100d90 2023-12-12T10:04:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f697f7fe700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f69680a15f0 0x7f69680a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f697f7fe700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f69680a15f0 0x7f69680a3ab0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6974005fd0 tx=0x7f6974009310 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f699239fb80 1 Processor -- start 2023-12-12T10:04:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f699239fb80 1 -- start start 2023-12-12T10:04:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f699239fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e81cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f699239fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e820d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f699239fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e821730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f699239fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e773af0 con 0x55cc4e8212d0 2023-12-12T10:04:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f699239fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e771410 con 0x55cc4e81cb80 2023-12-12T10:04:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.949+0000 7f699239fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e76cd00 con 0x55cc4e81d4a0 2023-12-12T10:04:00.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e821730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e81cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e81cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39842/0 (socket says 172.21.15.42:39842) 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e821730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36786/0 (socket says 172.21.15.42:36786) 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697f7fe700 1 -- 172.21.15.42:0/1026065101 learned_addr learned my addr 172.21.15.42:0/1026065101 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697effd700 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e820d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697f7fe700 1 -- 172.21.15.42:0/1026065101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 msgr2=0x55cc4e820d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697f7fe700 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e820d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697f7fe700 1 -- 172.21.15.42:0/1026065101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 msgr2=0x55cc4e821730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697f7fe700 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e821730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697f7fe700 1 -- 172.21.15.42:0/1026065101 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc4e5e95d0 con 0x55cc4e81cb80 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697ffff700 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e821730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697effd700 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e820d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f697f7fe700 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e81cf60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6974011d80 tx=0x7f6974014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9296f1549d2261d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f6965ffb700 1 -- 172.21.15.42:0/1026065101 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974004a30 con 0x55cc4e81cb80 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f6965ffb700 1 -- 172.21.15.42:0/1026065101 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69740054e0 con 0x55cc4e81cb80 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.950+0000 7f6965ffb700 1 -- 172.21.15.42:0/1026065101 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974003b40 con 0x55cc4e81cb80 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.951+0000 7f699239fb80 1 -- 172.21.15.42:0/1026065101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 msgr2=0x55cc4e81cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.951+0000 7f699239fb80 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e81cf60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6974011d80 tx=0x7f6974014000 comp rx=0 tx=0).stop 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.951+0000 7f699239fb80 1 -- 172.21.15.42:0/1026065101 shutdown_connections 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.951+0000 7f699239fb80 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e81cf60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.951+0000 7f699239fb80 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e821730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.951+0000 7f699239fb80 1 --2- 172.21.15.42:0/1026065101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e820d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.951+0000 7f699239fb80 1 -- 172.21.15.42:0/1026065101 >> 172.21.15.42:0/1026065101 conn(0x55cc4e747780 msgr2=0x55cc4e812c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.951+0000 7f699239fb80 1 -- 172.21.15.42:0/1026065101 shutdown_connections 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.951+0000 7f699239fb80 1 -- 172.21.15.42:0/1026065101 wait complete. 2023-12-12T10:04:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f699239fb80 1 -- start start 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f699239fb80 1 Processor -- start 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f699239fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81cb80 0x55cc4e828a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f699239fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e81d4a0 0x55cc4e825d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f699239fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e8212d0 0x55cc4e8264a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f699239fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e726500 con 0x55cc4e81d4a0 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f699239fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e7261a0 con 0x55cc4e8212d0 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f699239fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e5eb8c0 con 0x55cc4e81cb80 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f699239fb80 10 client.0 _ll_register_callbacks cb 0x55cc4e651a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f697effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e81d4a0 0x55cc4e825d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81cb80 0x55cc4e828a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f697effd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e81d4a0 0x55cc4e825d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36790/0 (socket says 172.21.15.42:36790) 2023-12-12T10:04:00.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f697f7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81cb80 0x55cc4e828a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51772/0 (socket says 172.21.15.42:51772) 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f697effd700 1 -- 172.21.15.42:0/3078925506 learned_addr learned my addr 172.21.15.42:0/3078925506 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.952+0000 7f697ffff700 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e8212d0 0x55cc4e8264a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f697effd700 1 -- 172.21.15.42:0/3078925506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81cb80 msgr2=0x55cc4e828a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f697effd700 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81cb80 0x55cc4e828a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f697effd700 1 -- 172.21.15.42:0/3078925506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e8212d0 msgr2=0x55cc4e8264a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f697effd700 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e8212d0 0x55cc4e8264a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f697effd700 1 -- 172.21.15.42:0/3078925506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698010dba0 con 0x55cc4e81d4a0 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f697ffff700 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e8212d0 0x55cc4e8264a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f697f7fe700 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81cb80 0x55cc4e828a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f697effd700 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e81d4a0 0x55cc4e825d60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f69700caaf0 tx=0x7f697009a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5673401f772206e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f695effd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f695effd700 1 -- 172.21.15.42:0/3078925506 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700a2660 con 0x55cc4e81d4a0 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f695effd700 1 -- 172.21.15.42:0/3078925506 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f69700cc040 con 0x55cc4e81d4a0 2023-12-12T10:04:00.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f699239fb80 1 -- 172.21.15.42:0/3078925506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc4e80b730 con 0x55cc4e81d4a0 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f695effd700 1 -- 172.21.15.42:0/3078925506 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69700cd280 con 0x55cc4e81d4a0 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f699239fb80 10 client.15981 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f699239fb80 1 -- 172.21.15.42:0/3078925506 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f6974017070 con 0x55cc4e81d4a0 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f695effd700 1 -- 172.21.15.42:0/3078925506 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc4e80b730 con 0x55cc4e81d4a0 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.953+0000 7f695effd700 1 -- 172.21.15.42:0/3078925506 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f69700a21d0 con 0x55cc4e81d4a0 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.954+0000 7f699239fb80 10 client.15981 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.954+0000 7f699239fb80 10 client.15981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.954+0000 7f699239fb80 20 client.15981 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.954+0000 7f699239fb80 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55cc4e836550 0x55cc4e838970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.954+0000 7f699239fb80 4 client.15981 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.954+0000 7f699239fb80 1 -- 172.21.15.42:0/3078925506 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc4e7e7ee0 con 0x55cc4e836550 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.954+0000 7f697f7fe700 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55cc4e836550 0x55cc4e838970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.954+0000 7f697f7fe700 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55cc4e836550 0x55cc4e838970 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80902226eb25e20b server_cookie=fee756fb1065aece in_seq=0 out_seq=0 2023-12-12T10:04:00.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.954+0000 7f695effd700 10 client.15981 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f695effd700 1 -- 172.21.15.42:0/3078925506 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc4e7e7ee0 con 0x55cc4e836550 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f695effd700 10 client.15981 handle_command_reply: tid=0 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 client.15981 shutdown 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 -- 172.21.15.42:0/3078925506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e81d4a0 msgr2=0x55cc4e825d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e81d4a0 0x55cc4e825d60 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f69700caaf0 tx=0x7f697009a350 comp rx=0 tx=0).stop 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 -- 172.21.15.42:0/3078925506 shutdown_connections 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55cc4e836550 0x55cc4e838970 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e8212d0 0x55cc4e8264a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81cb80 0x55cc4e828a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 --2- 172.21.15.42:0/3078925506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e81d4a0 0x55cc4e825d60 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f695effd700 0 client.15981 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 -- 172.21.15.42:0/3078925506 >> 172.21.15.42:0/3078925506 conn(0x55cc4e747780 msgr2=0x55cc4e817570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.955+0000 7f699239fb80 1 -- 172.21.15.42:0/3078925506 shutdown_connections 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.956+0000 7f699239fb80 1 -- 172.21.15.42:0/3078925506 wait complete. 2023-12-12T10:04:00.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.956+0000 7f699239fb80 20 client.15981 trim_cache size 0 max 16384 2023-12-12T10:04:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f699239fb80 1 Processor -- start 2023-12-12T10:04:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f699239fb80 1 -- start start 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f699239fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e825d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f699239fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e812c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f699239fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e8226c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f699239fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e825b50 con 0x55cc4e8212d0 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f699239fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e813e70 con 0x55cc4e81cb80 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f699239fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e833b00 con 0x55cc4e81d4a0 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f697ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e8226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f697effd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e812c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f697ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e8226c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36800/0 (socket says 172.21.15.42:36800) 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f697ffff700 1 -- 172.21.15.42:0/3573363763 learned_addr learned my addr 172.21.15.42:0/3573363763 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f697f7fe700 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e825d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f697ffff700 1 -- 172.21.15.42:0/3573363763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 msgr2=0x55cc4e812c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:00.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f697ffff700 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e812c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f697ffff700 1 -- 172.21.15.42:0/3573363763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 msgr2=0x55cc4e825d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.971+0000 7f697ffff700 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e825d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.972+0000 7f697ffff700 1 -- 172.21.15.42:0/3573363763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc4e595d80 con 0x55cc4e8212d0 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.972+0000 7f697effd700 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e812c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.972+0000 7f697f7fe700 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e825d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.972+0000 7f697ffff700 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e8226c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6978007b60 tx=0x7f6978007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43fa88bada8e0e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.972+0000 7f69667fc700 1 -- 172.21.15.42:0/3573363763 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697800e040 con 0x55cc4e8212d0 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.972+0000 7f69667fc700 1 -- 172.21.15.42:0/3573363763 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6978003cc0 con 0x55cc4e8212d0 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.972+0000 7f69667fc700 1 -- 172.21.15.42:0/3573363763 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6978005860 con 0x55cc4e8212d0 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.972+0000 7f699239fb80 1 -- 172.21.15.42:0/3573363763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 msgr2=0x55cc4e8226c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.972+0000 7f699239fb80 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e8226c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f6978007b60 tx=0x7f6978007b90 comp rx=0 tx=0).stop 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 -- 172.21.15.42:0/3573363763 shutdown_connections 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e825d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e8226c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 --2- 172.21.15.42:0/3573363763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e812c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 -- 172.21.15.42:0/3573363763 >> 172.21.15.42:0/3573363763 conn(0x55cc4e747780 msgr2=0x55cc4e817e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 -- 172.21.15.42:0/3573363763 shutdown_connections 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 -- 172.21.15.42:0/3573363763 wait complete. 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 -- start start 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 Processor -- start 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e829610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e829b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e88ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e82a210 con 0x55cc4e8212d0 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e88bf80 con 0x55cc4e81cb80 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc4e88c0c0 con 0x55cc4e81d4a0 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.973+0000 7f699239fb80 10 client.0 _ll_register_callbacks cb 0x55cc4e651a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e88ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697ffff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e88ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36806/0 (socket says 172.21.15.42:36806) 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697ffff700 1 -- 172.21.15.42:0/905859664 learned_addr learned my addr 172.21.15.42:0/905859664 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697effd700 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e829b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697f7fe700 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e829610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697ffff700 1 -- 172.21.15.42:0/905859664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 msgr2=0x55cc4e829b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697ffff700 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e829b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697ffff700 1 -- 172.21.15.42:0/905859664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 msgr2=0x55cc4e829610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697ffff700 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e829610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697ffff700 1 -- 172.21.15.42:0/905859664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6974017070 con 0x55cc4e8212d0 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f697ffff700 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e88ba40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6978007c00 tx=0x7f6978009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141a770cda926708 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f695effd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f695effd700 1 -- 172.21.15.42:0/905859664 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697800e050 con 0x55cc4e8212d0 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f695effd700 1 -- 172.21.15.42:0/905859664 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f69780056b0 con 0x55cc4e8212d0 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f699239fb80 1 -- 172.21.15.42:0/905859664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69740054e0 con 0x55cc4e8212d0 2023-12-12T10:04:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f695effd700 1 -- 172.21.15.42:0/905859664 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69780058b0 con 0x55cc4e8212d0 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f699239fb80 10 client.15987 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f699239fb80 1 -- 172.21.15.42:0/905859664 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7f69780078b0 con 0x55cc4e8212d0 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.974+0000 7f695effd700 1 -- 172.21.15.42:0/905859664 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69740054e0 con 0x55cc4e8212d0 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f695effd700 1 -- 172.21.15.42:0/905859664 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f697800b050 con 0x55cc4e8212d0 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f699239fb80 10 client.15987 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f699239fb80 10 client.15987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f699239fb80 20 client.15987 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f699239fb80 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55cc4e88d580 0x55cc4e88f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f699239fb80 4 client.15987 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f699239fb80 1 -- 172.21.15.42:0/905859664 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc4e732f70 con 0x55cc4e88d580 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f697f7fe700 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55cc4e88d580 0x55cc4e88f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f697f7fe700 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55cc4e88d580 0x55cc4e88f9a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed4dec7f7897126 server_cookie=52dfb40dedce91a7 in_seq=0 out_seq=0 2023-12-12T10:04:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:00.975+0000 7f695effd700 10 client.15987 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:00.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:00 smithi111 ceph-mon[102565]: pgmap v164: 129 pgs: 129 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.3 MiB/s wr, 732 op/s 2023-12-12T10:04:01.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.087+0000 7f695effd700 1 -- 172.21.15.42:0/905859664 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc4e732f70 con 0x55cc4e88d580 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.087+0000 7f695effd700 10 client.15987 handle_command_reply: tid=0 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.087+0000 7f699239fb80 1 client.15987 shutdown 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 -- 172.21.15.42:0/905859664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 msgr2=0x55cc4e88ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e88ba40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6978007c00 tx=0x7f6978009b20 comp rx=0 tx=0).stop 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 -- 172.21.15.42:0/905859664 shutdown_connections 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55cc4e88d580 0x55cc4e88f9a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55cc4e81cb80 0x55cc4e829610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cc4e8212d0 0x55cc4e88ba40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f695effd700 0 client.15987 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 --2- 172.21.15.42:0/905859664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55cc4e81d4a0 0x55cc4e829b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:01.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 -- 172.21.15.42:0/905859664 >> 172.21.15.42:0/905859664 conn(0x55cc4e747780 msgr2=0x55cc4e8236c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 -- 172.21.15.42:0/905859664 shutdown_connections 2023-12-12T10:04:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 1 -- 172.21.15.42:0/905859664 wait complete. 2023-12-12T10:04:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f699239fb80 20 client.15987 trim_cache size 0 max 16384 2023-12-12T10:04:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:04:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:04:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:04:01.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:04:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f69861a6700 1 -- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f69680a15f0 msgr2=0x7f69680a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f69861a6700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f69680a15f0 0x7f69680a3ab0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6974005fd0 tx=0x7f6974009310 comp rx=0 tx=0).stop 2023-12-12T10:04:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f69861a6700 1 -- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 msgr2=0x7f6980198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.088+0000 7f69861a6700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 0x7f6980198bf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f697000e9a0 tx=0x7f697000ecb0 comp rx=0 tx=0).stop 2023-12-12T10:04:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.089+0000 7f69861a6700 1 -- 172.21.15.42:0/3462912144 shutdown_connections 2023-12-12T10:04:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.089+0000 7f69861a6700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f69680a15f0 0x7f69680a3ab0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.089+0000 7f69861a6700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f69801003d0 0x7f69801986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.089+0000 7f69861a6700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6980106470 0x7f69801a3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:01.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.089+0000 7f69861a6700 1 --2- 172.21.15.42:0/3462912144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6980100d90 0x7f6980198bf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.089+0000 7f69861a6700 1 -- 172.21.15.42:0/3462912144 >> 172.21.15.42:0/3462912144 conn(0x7f69800fbea0 msgr2=0x7f69800fc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.089+0000 7f69861a6700 1 -- 172.21.15.42:0/3462912144 shutdown_connections 2023-12-12T10:04:01.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:01.089+0000 7f69861a6700 1 -- 172.21.15.42:0/3462912144 wait complete. 2023-12-12T10:04:02.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:02 smithi173 ceph-mon[103308]: pgmap v165: 129 pgs: 129 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 439 op/s 2023-12-12T10:04:02.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:02 smithi042 ceph-mon[98894]: pgmap v165: 129 pgs: 129 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 439 op/s 2023-12-12T10:04:02.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:02 smithi111 ceph-mon[102565]: pgmap v165: 129 pgs: 129 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 439 op/s 2023-12-12T10:04:04.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:04 smithi173 ceph-mon[103308]: pgmap v166: 129 pgs: 129 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 457 op/s 2023-12-12T10:04:04.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:04 smithi042 ceph-mon[98894]: pgmap v166: 129 pgs: 129 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 457 op/s 2023-12-12T10:04:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:04 smithi111 ceph-mon[102565]: pgmap v166: 129 pgs: 129 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 457 op/s 2023-12-12T10:04:06.162 INFO:tasks.workunit.client.0.smithi042.stdout:cleaned up hierarchy 2023-12-12T10:04:06.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:06 smithi173 ceph-mon[103308]: pgmap v167: 129 pgs: 129 active+clean; 570 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 682 op/s 2023-12-12T10:04:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:06 smithi042 ceph-mon[98894]: pgmap v167: 129 pgs: 129 active+clean; 570 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 682 op/s 2023-12-12T10:04:06.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:06 smithi111 ceph-mon[102565]: pgmap v167: 129 pgs: 129 active+clean; 570 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 4.4 MiB/s wr, 682 op/s 2023-12-12T10:04:08.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:08 smithi173 ceph-mon[103308]: pgmap v168: 129 pgs: 129 active+clean; 570 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 445 op/s 2023-12-12T10:04:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:08 smithi042 ceph-mon[98894]: pgmap v168: 129 pgs: 129 active+clean; 570 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 445 op/s 2023-12-12T10:04:08.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:08 smithi111 ceph-mon[102565]: pgmap v168: 129 pgs: 129 active+clean; 570 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 445 op/s 2023-12-12T10:04:10.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:10 smithi173 ceph-mon[103308]: pgmap v169: 129 pgs: 129 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 674 op/s 2023-12-12T10:04:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:10 smithi042 ceph-mon[98894]: pgmap v169: 129 pgs: 129 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 674 op/s 2023-12-12T10:04:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:10 smithi111 ceph-mon[102565]: pgmap v169: 129 pgs: 129 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.2 MiB/s wr, 674 op/s 2023-12-12T10:04:12.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:12 smithi173 ceph-mon[103308]: pgmap v170: 129 pgs: 129 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 482 op/s 2023-12-12T10:04:12.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:12 smithi042 ceph-mon[98894]: pgmap v170: 129 pgs: 129 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 482 op/s 2023-12-12T10:04:12.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:12 smithi111 ceph-mon[102565]: pgmap v170: 129 pgs: 129 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.9 MiB/s wr, 482 op/s 2023-12-12T10:04:14.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.226+0000 7f6496489700 1 -- 172.21.15.42:0/1164495809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6490068ca0 msgr2=0x7f6490069080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.226+0000 7f6496489700 1 --2- 172.21.15.42:0/1164495809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6490068ca0 0x7f6490069080 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6480009990 tx=0x7f6480009ca0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.226+0000 7f6496489700 1 -- 172.21.15.42:0/1164495809 shutdown_connections 2023-12-12T10:04:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.226+0000 7f6496489700 1 --2- 172.21.15.42:0/1164495809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6490106a40 0x7f649010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.226+0000 7f6496489700 1 --2- 172.21.15.42:0/1164495809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6490068ca0 0x7f6490069080 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.226+0000 7f6496489700 1 --2- 172.21.15.42:0/1164495809 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6490106080 0x7f6490106500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.226+0000 7f6496489700 1 -- 172.21.15.42:0/1164495809 >> 172.21.15.42:0/1164495809 conn(0x7f6490077570 msgr2=0x7f6490077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.226+0000 7f6496489700 1 -- 172.21.15.42:0/1164495809 shutdown_connections 2023-12-12T10:04:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.226+0000 7f6496489700 1 -- 172.21.15.42:0/1164495809 wait complete. 2023-12-12T10:04:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.227+0000 7f6496489700 1 Processor -- start 2023-12-12T10:04:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.227+0000 7f6496489700 1 -- start start 2023-12-12T10:04:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.227+0000 7f6496489700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6490068ca0 0x7f6490100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.227+0000 7f6496489700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6490106080 0x7f6490100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.227+0000 7f6496489700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6490106a40 0x7f6490100ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.227+0000 7f6496489700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64901a5d70 con 0x7f6490106080 2023-12-12T10:04:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.227+0000 7f6496489700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64901a5ee0 con 0x7f6490068ca0 2023-12-12T10:04:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.227+0000 7f6496489700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64901a60a0 con 0x7f6490106a40 2023-12-12T10:04:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f648ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6490068ca0 0x7f6490100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f648ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6490068ca0 0x7f6490100440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39876/0 (socket says 172.21.15.42:39876) 2023-12-12T10:04:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f648f7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6490106080 0x7f6490100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f648ffff700 1 -- 172.21.15.42:0/2305204705 learned_addr learned my addr 172.21.15.42:0/2305204705 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f6494a26700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6490106a40 0x7f6490100ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f6494a26700 1 -- 172.21.15.42:0/2305204705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6490068ca0 msgr2=0x7f6490100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f6494a26700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6490068ca0 0x7f6490100440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f6494a26700 1 -- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6490106080 msgr2=0x7f6490100980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f6494a26700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6490106080 0x7f6490100980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f6494a26700 1 -- 172.21.15.42:0/2305204705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64800095f0 con 0x7f6490106a40 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f648f7fe700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6490106080 0x7f6490100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.228+0000 7f6494a26700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6490106a40 0x7f6490100ec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f648800efc0 tx=0x7f648800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.229+0000 7f648d7fa700 1 -- 172.21.15.42:0/2305204705 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6488018070 con 0x7f6490106a40 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.229+0000 7f6496489700 1 -- 172.21.15.42:0/2305204705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64901a6320 con 0x7f6490106a40 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.229+0000 7f6496489700 1 -- 172.21.15.42:0/2305204705 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64901a6830 con 0x7f6490106a40 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.229+0000 7f648d7fa700 1 -- 172.21.15.42:0/2305204705 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f648800f040 con 0x7f6490106a40 2023-12-12T10:04:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.229+0000 7f648d7fa700 1 -- 172.21.15.42:0/2305204705 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648800cd60 con 0x7f6490106a40 2023-12-12T10:04:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.232+0000 7f648d7fa700 1 -- 172.21.15.42:0/2305204705 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f6488004ad0 con 0x7f6490106a40 2023-12-12T10:04:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.233+0000 7f648d7fa700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f64780a1670 0x7f64780a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.233+0000 7f648d7fa700 1 -- 172.21.15.42:0/2305204705 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f64880ccd60 con 0x7f6490106a40 2023-12-12T10:04:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.234+0000 7f648ffff700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f64780a1670 0x7f64780a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.234+0000 7f64a2682b80 1 Processor -- start 2023-12-12T10:04:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.234+0000 7f64a2682b80 1 -- start start 2023-12-12T10:04:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee4021930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021e70 0x555ee401bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648ffff700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f64780a1670 0x7f64780a3b30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6480006010 tx=0x7f648000b260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee401c320 0x555ee401c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f64a2682b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee3f6eaf0 con 0x555ee401c320 2023-12-12T10:04:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f64a2682b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee3f6c410 con 0x555ee4021e70 2023-12-12T10:04:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f64a2682b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee3f67d00 con 0x555ee4021550 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648f7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021e70 0x555ee401bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee4021930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648f7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021e70 0x555ee401bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39888/0 (socket says 172.21.15.42:39888) 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee4021930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51808/0 (socket says 172.21.15.42:51808) 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648f7fe700 1 -- 172.21.15.42:0/908555103 learned_addr learned my addr 172.21.15.42:0/908555103 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648f7fe700 1 -- 172.21.15.42:0/908555103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 msgr2=0x555ee4021930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648f7fe700 1 --2- 172.21.15.42:0/908555103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee4021930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648f7fe700 1 -- 172.21.15.42:0/908555103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee401c320 msgr2=0x555ee401c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648f7fe700 1 --2- 172.21.15.42:0/908555103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee401c320 0x555ee401c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.235+0000 7f648f7fe700 1 -- 172.21.15.42:0/908555103 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ee3de45d0 con 0x555ee4021e70 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.236+0000 7f648f7fe700 1 --2- 172.21.15.42:0/908555103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021e70 0x555ee401bd90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64840115f0 tx=0x7f6484011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e992f9733a25b5f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.237+0000 7f64767fc700 1 -- 172.21.15.42:0/908555103 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484004230 con 0x555ee4021e70 2023-12-12T10:04:14.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.237+0000 7f64767fc700 1 -- 172.21.15.42:0/908555103 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6484004390 con 0x555ee4021e70 2023-12-12T10:04:14.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.237+0000 7f64767fc700 1 -- 172.21.15.42:0/908555103 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6484019890 con 0x555ee4021e70 2023-12-12T10:04:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.238+0000 7f64a2682b80 1 -- 172.21.15.42:0/908555103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021e70 msgr2=0x555ee401bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.238+0000 7f64a2682b80 1 --2- 172.21.15.42:0/908555103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021e70 0x555ee401bd90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64840115f0 tx=0x7f6484011900 comp rx=0 tx=0).stop 2023-12-12T10:04:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.238+0000 7f64a2682b80 1 -- 172.21.15.42:0/908555103 shutdown_connections 2023-12-12T10:04:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.238+0000 7f64a2682b80 1 --2- 172.21.15.42:0/908555103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee401c320 0x555ee401c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.238+0000 7f64a2682b80 1 --2- 172.21.15.42:0/908555103 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee4021930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.238+0000 7f64a2682b80 1 --2- 172.21.15.42:0/908555103 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021e70 0x555ee401bd90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.238+0000 7f64a2682b80 1 -- 172.21.15.42:0/908555103 >> 172.21.15.42:0/908555103 conn(0x555ee3f42780 msgr2=0x555ee400dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:14.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.238+0000 7f64a2682b80 1 -- 172.21.15.42:0/908555103 shutdown_connections 2023-12-12T10:04:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.238+0000 7f64a2682b80 1 -- 172.21.15.42:0/908555103 wait complete. 2023-12-12T10:04:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.239+0000 7f64a2682b80 1 -- start start 2023-12-12T10:04:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.239+0000 7f64a2682b80 1 Processor -- start 2023-12-12T10:04:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.239+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 0x555ee4025250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.239+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021550 0x555ee4025790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.239+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021e70 0x555ee4025cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.239+0000 7f64a2682b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee3f21500 con 0x555ee4021550 2023-12-12T10:04:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.239+0000 7f64a2682b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee3f211a0 con 0x555ee401c320 2023-12-12T10:04:14.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.239+0000 7f64a2682b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee3de68c0 con 0x555ee4021e70 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.239+0000 7f64a2682b80 10 client.0 _ll_register_callbacks cb 0x555ee3e4ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f6494a26700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021e70 0x555ee4025cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f6494a26700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021e70 0x555ee4025cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51814/0 (socket says 172.21.15.42:51814) 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f6494a26700 1 -- 172.21.15.42:0/2569473607 learned_addr learned my addr 172.21.15.42:0/2569473607 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f648f7fe700 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021550 0x555ee4025790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f648ffff700 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 0x555ee4025250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f6494a26700 1 -- 172.21.15.42:0/2569473607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 msgr2=0x555ee4025250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f6494a26700 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 0x555ee4025250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f6494a26700 1 -- 172.21.15.42:0/2569473607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021550 msgr2=0x555ee4025790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f6494a26700 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021550 0x555ee4025790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f6494a26700 1 -- 172.21.15.42:0/2569473607 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6484009d50 con 0x555ee4021e70 2023-12-12T10:04:14.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f6494a26700 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021e70 0x555ee4025cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64880ca760 tx=0x7f64880996b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5f7d3724ada32fa server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.240+0000 7f646f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2023-12-12T10:04:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.241+0000 7f646f7fe700 1 -- 172.21.15.42:0/2569473607 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648809a180 con 0x555ee4021e70 2023-12-12T10:04:14.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.241+0000 7f646f7fe700 1 -- 172.21.15.42:0/2569473607 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6488099a80 con 0x555ee4021e70 2023-12-12T10:04:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.241+0000 7f64a2682b80 1 -- 172.21.15.42:0/2569473607 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555ee4006730 con 0x555ee4021e70 2023-12-12T10:04:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.241+0000 7f646f7fe700 1 -- 172.21.15.42:0/2569473607 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f648809dbd0 con 0x555ee4021e70 2023-12-12T10:04:14.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f64a2682b80 10 client.25627 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f64a2682b80 1 -- 172.21.15.42:0/2569473607 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555ee3d90d80 con 0x555ee4021e70 2023-12-12T10:04:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f646f7fe700 1 -- 172.21.15.42:0/2569473607 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555ee4006730 con 0x555ee4021e70 2023-12-12T10:04:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f646f7fe700 1 -- 172.21.15.42:0/2569473607 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f64880b91c0 con 0x555ee4021e70 2023-12-12T10:04:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f64a2682b80 10 client.25627 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f64a2682b80 10 client.25627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f64a2682b80 20 client.25627 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:14.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f64a2682b80 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x555ee4035a50 0x555ee4037e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f64a2682b80 4 client.25627 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:04:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.242+0000 7f64a2682b80 1 -- 172.21.15.42:0/2569473607 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555ee3fe2ee0 con 0x555ee4035a50 2023-12-12T10:04:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.243+0000 7f648ffff700 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x555ee4035a50 0x555ee4037e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.243+0000 7f646f7fe700 10 client.25627 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.243+0000 7f648ffff700 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x555ee4035a50 0x555ee4037e70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b9204d610c6082 server_cookie=2a91458ca496f76 in_seq=0 out_seq=0 2023-12-12T10:04:14.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f646f7fe700 1 -- 172.21.15.42:0/2569473607 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555ee3fe2ee0 con 0x555ee4035a50 2023-12-12T10:04:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f646f7fe700 10 client.25627 handle_command_reply: tid=0 2023-12-12T10:04:14.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 client.25627 shutdown 2023-12-12T10:04:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 -- 172.21.15.42:0/2569473607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021e70 msgr2=0x555ee4025cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021e70 0x555ee4025cd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f64880ca760 tx=0x7f64880996b0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 -- 172.21.15.42:0/2569473607 shutdown_connections 2023-12-12T10:04:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x555ee4035a50 0x555ee4037e70 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 0x555ee4025250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f646f7fe700 0 client.25627 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021e70 0x555ee4025cd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 --2- 172.21.15.42:0/2569473607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021550 0x555ee4025790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 -- 172.21.15.42:0/2569473607 >> 172.21.15.42:0/2569473607 conn(0x555ee3f42780 msgr2=0x555ee4012570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 -- 172.21.15.42:0/2569473607 shutdown_connections 2023-12-12T10:04:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 1 -- 172.21.15.42:0/2569473607 wait complete. 2023-12-12T10:04:14.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.244+0000 7f64a2682b80 20 client.25627 trim_cache size 0 max 16384 2023-12-12T10:04:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f64a2682b80 1 Processor -- start 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f64a2682b80 1 -- start start 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 0x555ee4012c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee401c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 0x555ee4024b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f64a2682b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee40232d0 con 0x555ee4021e70 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f64a2682b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee403d060 con 0x555ee401c320 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f64a2682b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee40237a0 con 0x555ee4021550 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f6494a26700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 0x555ee4024b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f648f7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee401c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f648f7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee401c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51824/0 (socket says 172.21.15.42:51824) 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f6494a26700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 0x555ee4024b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36842/0 (socket says 172.21.15.42:36842) 2023-12-12T10:04:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f648f7fe700 1 -- 172.21.15.42:0/3005686680 learned_addr learned my addr 172.21.15.42:0/3005686680 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f648ffff700 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 0x555ee4012c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f648ffff700 1 -- 172.21.15.42:0/3005686680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 msgr2=0x555ee401c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f648ffff700 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee401c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f648ffff700 1 -- 172.21.15.42:0/3005686680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 msgr2=0x555ee4024b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.267+0000 7f648ffff700 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 0x555ee4024b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.268+0000 7f648ffff700 1 -- 172.21.15.42:0/3005686680 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555ee3d79c00 con 0x555ee401c320 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.268+0000 7f648f7fe700 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee401c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.268+0000 7f6494a26700 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 0x555ee4024b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.268+0000 7f648ffff700 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 0x555ee4012c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x555ee4038780 tx=0x7f6480020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be71b03f3a104165 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.268+0000 7f6476ffd700 1 -- 172.21.15.42:0/3005686680 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800350f0 con 0x555ee401c320 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.268+0000 7f6476ffd700 1 -- 172.21.15.42:0/3005686680 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f648001e040 con 0x555ee401c320 2023-12-12T10:04:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.268+0000 7f6476ffd700 1 -- 172.21.15.42:0/3005686680 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480004b30 con 0x555ee401c320 2023-12-12T10:04:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 -- 172.21.15.42:0/3005686680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 msgr2=0x555ee4012c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 0x555ee4012c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x555ee4038780 tx=0x7f6480020000 comp rx=0 tx=0).stop 2023-12-12T10:04:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 -- 172.21.15.42:0/3005686680 shutdown_connections 2023-12-12T10:04:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee401c320 0x555ee4012c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 0x555ee4024b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3005686680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee4021550 0x555ee401c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 -- 172.21.15.42:0/3005686680 >> 172.21.15.42:0/3005686680 conn(0x555ee3f42780 msgr2=0x555ee4012570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 -- 172.21.15.42:0/3005686680 shutdown_connections 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 -- 172.21.15.42:0/3005686680 wait complete. 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 -- start start 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.269+0000 7f64a2682b80 1 Processor -- start 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee401c320 0x555ee402b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021550 0x555ee402ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f64a2682b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 0x555ee402bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f64a2682b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee408bf60 con 0x555ee4021e70 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f64a2682b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee408c0a0 con 0x555ee4021550 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f64a2682b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x555ee408c230 con 0x555ee401c320 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f64a2682b80 10 client.0 _ll_register_callbacks cb 0x555ee3e4ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648f7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021550 0x555ee402ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee401c320 0x555ee402b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648f7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021550 0x555ee402ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39906/0 (socket says 172.21.15.42:39906) 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648f7fe700 1 -- 172.21.15.42:0/3752747207 learned_addr learned my addr 172.21.15.42:0/3752747207 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648ffff700 1 -- 172.21.15.42:0/3752747207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021550 msgr2=0x555ee402ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648ffff700 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021550 0x555ee402ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648ffff700 1 -- 172.21.15.42:0/3752747207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 msgr2=0x555ee402bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648ffff700 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 0x555ee402bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648ffff700 1 -- 172.21.15.42:0/3752747207 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555ee3de45d0 con 0x555ee401c320 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.270+0000 7f648f7fe700 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021550 0x555ee402ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f648ffff700 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee401c320 0x555ee402b4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6480005f50 tx=0x7f6480005a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ea9eb283074c8d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f646f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.173:3300/0 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f646f7fe700 1 -- 172.21.15.42:0/3752747207 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480004990 con 0x555ee401c320 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f646f7fe700 1 -- 172.21.15.42:0/3752747207 <== mon.1 v2:172.21.15.173:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6480004af0 con 0x555ee401c320 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f64a2682b80 1 -- 172.21.15.42:0/3752747207 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6484004390 con 0x555ee401c320 2023-12-12T10:04:14.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f646f7fe700 1 -- 172.21.15.42:0/3752747207 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6480004cb0 con 0x555ee401c320 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f64a2682b80 10 client.25633 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f64a2682b80 1 -- 172.21.15.42:0/3752747207 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x555ee3d90d80 con 0x555ee401c320 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f646f7fe700 1 -- 172.21.15.42:0/3752747207 <== mon.1 v2:172.21.15.173:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6484004390 con 0x555ee401c320 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f646f7fe700 1 -- 172.21.15.42:0/3752747207 <== mon.1 v2:172.21.15.173:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f648000f580 con 0x555ee401c320 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f64a2682b80 10 client.25633 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f64a2682b80 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f64a2682b80 20 client.25633 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x555ee408d720 0x555ee408fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.271+0000 7f64a2682b80 4 client.25633 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.272+0000 7f64a2682b80 1 -- 172.21.15.42:0/3752747207 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555ee3f2df70 con 0x555ee408d720 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.272+0000 7f648f7fe700 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x555ee408d720 0x555ee408fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.272+0000 7f646f7fe700 10 client.25633 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:14.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.272+0000 7f648f7fe700 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x555ee408d720 0x555ee408fb40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60f5901c2fbba7cb server_cookie=889fac02332f1643 in_seq=0 out_seq=0 2023-12-12T10:04:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f646f7fe700 1 -- 172.21.15.42:0/3752747207 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555ee3f2df70 con 0x555ee408d720 2023-12-12T10:04:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f646f7fe700 10 client.25633 handle_command_reply: tid=0 2023-12-12T10:04:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 client.25633 shutdown 2023-12-12T10:04:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 -- 172.21.15.42:0/3752747207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee401c320 msgr2=0x555ee402b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee401c320 0x555ee402b4c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6480005f50 tx=0x7f6480005a90 comp rx=0 tx=0).stop 2023-12-12T10:04:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 -- 172.21.15.42:0/3752747207 shutdown_connections 2023-12-12T10:04:14.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555ee4021e70 0x555ee402bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x555ee401c320 0x555ee402b4c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x555ee408d720 0x555ee408fb40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 --2- 172.21.15.42:0/3752747207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x555ee4021550 0x555ee402ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f646f7fe700 0 client.25633 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 -- 172.21.15.42:0/3752747207 >> 172.21.15.42:0/3752747207 conn(0x555ee3f42780 msgr2=0x555ee401efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 -- 172.21.15.42:0/3752747207 shutdown_connections 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.487+0000 7f64a2682b80 1 -- 172.21.15.42:0/3752747207 wait complete. 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f64a2682b80 20 client.25633 trim_cache size 0 max 16384 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:04:14.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:04:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:04:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 -- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f64780a1670 msgr2=0x7f64780a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f64780a1670 0x7f64780a3b30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6480006010 tx=0x7f648000b260 comp rx=0 tx=0).stop 2023-12-12T10:04:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 -- 172.21.15.42:0/2305204705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6490106a40 msgr2=0x7f6490100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:14.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6490106a40 0x7f6490100ec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f648800efc0 tx=0x7f648800c3a0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 -- 172.21.15.42:0/2305204705 shutdown_connections 2023-12-12T10:04:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f64780a1670 0x7f64780a3b30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6490068ca0 0x7f6490100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6490106a40 0x7f6490100ec0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 --2- 172.21.15.42:0/2305204705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6490106080 0x7f6490100980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 -- 172.21.15.42:0/2305204705 >> 172.21.15.42:0/2305204705 conn(0x7f6490077570 msgr2=0x7f64900fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 -- 172.21.15.42:0/2305204705 shutdown_connections 2023-12-12T10:04:14.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:14.488+0000 7f6496489700 1 -- 172.21.15.42:0/2305204705 wait complete. 2023-12-12T10:04:14.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:14 smithi173 ceph-mon[103308]: pgmap v171: 129 pgs: 129 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 493 op/s 2023-12-12T10:04:14.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:14 smithi042 ceph-mon[98894]: pgmap v171: 129 pgs: 129 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 493 op/s 2023-12-12T10:04:14.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:14 smithi111 ceph-mon[102565]: pgmap v171: 129 pgs: 129 active+clean; 578 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 493 op/s 2023-12-12T10:04:16.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:16 smithi173 ceph-mon[103308]: pgmap v172: 129 pgs: 129 active+clean; 584 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 579 op/s 2023-12-12T10:04:16.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:16 smithi042 ceph-mon[98894]: pgmap v172: 129 pgs: 129 active+clean; 584 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 579 op/s 2023-12-12T10:04:16.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:16 smithi111 ceph-mon[102565]: pgmap v172: 129 pgs: 129 active+clean; 584 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.5 MiB/s wr, 579 op/s 2023-12-12T10:04:18.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:18 smithi173 ceph-mon[103308]: pgmap v173: 129 pgs: 129 active+clean; 584 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2023-12-12T10:04:18.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:18 smithi042 ceph-mon[98894]: pgmap v173: 129 pgs: 129 active+clean; 584 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2023-12-12T10:04:18.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:18 smithi111 ceph-mon[102565]: pgmap v173: 129 pgs: 129 active+clean; 584 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.2 MiB/s wr, 343 op/s 2023-12-12T10:04:20.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:20 smithi173 ceph-mon[103308]: pgmap v174: 129 pgs: 129 active+clean; 618 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 598 op/s 2023-12-12T10:04:20.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:20 smithi042 ceph-mon[98894]: pgmap v174: 129 pgs: 129 active+clean; 618 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 598 op/s 2023-12-12T10:04:20.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:20 smithi111 ceph-mon[102565]: pgmap v174: 129 pgs: 129 active+clean; 618 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 598 op/s 2023-12-12T10:04:22.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:22 smithi173 ceph-mon[103308]: pgmap v175: 129 pgs: 129 active+clean; 618 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 369 op/s 2023-12-12T10:04:22.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:22 smithi042 ceph-mon[98894]: pgmap v175: 129 pgs: 129 active+clean; 618 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 369 op/s 2023-12-12T10:04:22.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:22 smithi111 ceph-mon[102565]: pgmap v175: 129 pgs: 129 active+clean; 618 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 3.7 MiB/s wr, 369 op/s 2023-12-12T10:04:24.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:24 smithi173 ceph-mon[103308]: pgmap v176: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 385 op/s 2023-12-12T10:04:24.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:24 smithi042 ceph-mon[98894]: pgmap v176: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 385 op/s 2023-12-12T10:04:24.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:24 smithi111 ceph-mon[102565]: pgmap v176: 129 pgs: 129 active+clean; 622 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.1 MiB/s wr, 385 op/s 2023-12-12T10:04:26.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:26 smithi173 ceph-mon[103308]: pgmap v177: 129 pgs: 129 active+clean; 652 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.6 MiB/s wr, 499 op/s 2023-12-12T10:04:26.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:26 smithi042 ceph-mon[98894]: pgmap v177: 129 pgs: 129 active+clean; 652 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.6 MiB/s wr, 499 op/s 2023-12-12T10:04:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:26 smithi111 ceph-mon[102565]: pgmap v177: 129 pgs: 129 active+clean; 652 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.6 MiB/s wr, 499 op/s 2023-12-12T10:04:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.606+0000 7ff5c18fe700 1 -- 172.21.15.42:0/2150378716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc1003d0 msgr2=0x7ff5bc100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.606+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/2150378716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc1003d0 0x7ff5bc100850 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009990 tx=0x7ff5b0009ca0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.606+0000 7ff5c18fe700 1 -- 172.21.15.42:0/2150378716 shutdown_connections 2023-12-12T10:04:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.606+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/2150378716 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5bc100d90 0x7ff5bc10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.606+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/2150378716 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc106480 0x7ff5bc106860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.606+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/2150378716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc1003d0 0x7ff5bc100850 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.606+0000 7ff5c18fe700 1 -- 172.21.15.42:0/2150378716 >> 172.21.15.42:0/2150378716 conn(0x7ff5bc077570 msgr2=0x7ff5bc077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.606+0000 7ff5c18fe700 1 -- 172.21.15.42:0/2150378716 shutdown_connections 2023-12-12T10:04:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.606+0000 7ff5c18fe700 1 -- 172.21.15.42:0/2150378716 wait complete. 2023-12-12T10:04:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5c18fe700 1 Processor -- start 2023-12-12T10:04:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5c18fe700 1 -- start start 2023-12-12T10:04:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5c18fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc1003d0 0x7ff5bc102070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5c18fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc100d90 0x7ff5bc1025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5c18fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5bc106480 0x7ff5bc102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5c18fe700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc1a5d30 con 0x7ff5bc100d90 2023-12-12T10:04:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5c18fe700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc1a5ea0 con 0x7ff5bc106480 2023-12-12T10:04:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5c18fe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc1a6060 con 0x7ff5bc1003d0 2023-12-12T10:04:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc100d90 0x7ff5bc1025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.607+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc100d90 0x7ff5bc1025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36862/0 (socket says 172.21.15.42:36862) 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/577427459 learned_addr learned my addr 172.21.15.42:0/577427459 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5baffd700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc1003d0 0x7ff5bc102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5bb7fe700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5bc106480 0x7ff5bc102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/577427459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc1003d0 msgr2=0x7ff5bc102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc1003d0 0x7ff5bc102070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/577427459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5bc106480 msgr2=0x7ff5bc102af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5bc106480 0x7ff5bc102af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/577427459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00095f0 con 0x7ff5bc100d90 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc100d90 0x7ff5bc1025b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0000c00 tx=0x7ff5b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5c08fc700 1 -- 172.21.15.42:0/577427459 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b001d070 con 0x7ff5bc100d90 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5c18fe700 1 -- 172.21.15.42:0/577427459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc1a62e0 con 0x7ff5bc100d90 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5c08fc700 1 -- 172.21.15.42:0/577427459 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b0022470 con 0x7ff5bc100d90 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5c08fc700 1 -- 172.21.15.42:0/577427459 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0017650 con 0x7ff5bc100d90 2023-12-12T10:04:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.608+0000 7ff5c18fe700 1 -- 172.21.15.42:0/577427459 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc1a67e0 con 0x7ff5bc100d90 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.611+0000 7ff5c08fc700 1 -- 172.21.15.42:0/577427459 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7ff5b00225e0 con 0x7ff5bc100d90 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.612+0000 7ff5c08fc700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7ff5a40a15f0 0x7ff5a40a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.612+0000 7ff5c08fc700 1 -- 172.21.15.42:0/577427459 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff5b00ce5f0 con 0x7ff5bc100d90 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.612+0000 7ff5baffd700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7ff5a40a15f0 0x7ff5a40a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.612+0000 7ff5baffd700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7ff5a40a15f0 0x7ff5a40a3ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009be0 tx=0x7ff5ac009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5cdaf7b80 1 Processor -- start 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5cdaf7b80 1 -- start start 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7728d40 0x55acf7729120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7729660 0x55acf7723580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7723f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf767eaf0 con 0x55acf7729660 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf767c410 con 0x55acf7723ac0 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf7677d00 con 0x55acf7728d40 2023-12-12T10:04:27.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7729660 0x55acf7723580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5baffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7728d40 0x55acf7729120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7729660 0x55acf7723580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36870/0 (socket says 172.21.15.42:36870) 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/399433869 learned_addr learned my addr 172.21.15.42:0/399433869 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5bb7fe700 1 --2- 172.21.15.42:0/399433869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7723f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/399433869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7728d40 msgr2=0x55acf7729120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/399433869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7728d40 0x55acf7729120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/399433869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 msgr2=0x55acf7723f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/399433869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7723f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/399433869 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf74f45d0 con 0x55acf7729660 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.613+0000 7ff5bb7fe700 1 --2- 172.21.15.42:0/399433869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7723f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.614+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/399433869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7729660 0x55acf7723580 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006010 tx=0x7ff5b009aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9837f888a46025e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.614+0000 7ff5a1ffb700 1 -- 172.21.15.42:0/399433869 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b009b310 con 0x55acf7729660 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.614+0000 7ff5a1ffb700 1 -- 172.21.15.42:0/399433869 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b00cd070 con 0x55acf7729660 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.614+0000 7ff5a1ffb700 1 -- 172.21.15.42:0/399433869 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b009f220 con 0x55acf7729660 2023-12-12T10:04:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/399433869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7729660 msgr2=0x55acf7723580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/399433869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7729660 0x55acf7723580 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0006010 tx=0x7ff5b009aed0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/399433869 shutdown_connections 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/399433869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7723f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/399433869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7728d40 0x55acf7729120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/399433869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7729660 0x55acf7723580 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/399433869 >> 172.21.15.42:0/399433869 conn(0x55acf7652780 msgr2=0x55acf771dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/399433869 shutdown_connections 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/399433869 wait complete. 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 -- start start 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.615+0000 7ff5cdaf7b80 1 Processor -- start 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7723ac0 0x55acf772f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7728d40 0x55acf772c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 0x55acf772d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf7631500 con 0x55acf7723ac0 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf76311a0 con 0x55acf7728d40 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf74f68c0 con 0x55acf7729660 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5cdaf7b80 10 client.0 _ll_register_callbacks cb 0x55acf755ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7728d40 0x55acf772c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7728d40 0x55acf772c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:39936/0 (socket says 172.21.15.42:39936) 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/4181738786 learned_addr learned my addr 172.21.15.42:0/4181738786 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5bb7fe700 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 0x55acf772d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/4181738786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 msgr2=0x55acf772d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 0x55acf772d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/4181738786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7723ac0 msgr2=0x55acf772f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7723ac0 0x55acf772f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/4181738786 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b4011230 con 0x55acf7728d40 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7728d40 0x55acf772c930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005f50 tx=0x7ff5b0005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4a19353c51c0d4a server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.616+0000 7ff59affd700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff59affd700 1 -- 172.21.15.42:0/4181738786 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b009b310 con 0x55acf7728d40 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff59affd700 1 -- 172.21.15.42:0/4181738786 <== mon.2 v2:172.21.15.111:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff5b009f460 con 0x55acf7728d40 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/4181738786 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acf7716730 con 0x55acf7728d40 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff59affd700 1 -- 172.21.15.42:0/4181738786 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5b009f120 con 0x55acf7728d40 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff5cdaf7b80 10 client.25703 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/4181738786 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7ff5bc1a67e0 con 0x55acf7728d40 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff59affd700 1 -- 172.21.15.42:0/4181738786 <== mon.2 v2:172.21.15.111:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b00a7080 con 0x55acf7728d40 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff59affd700 1 -- 172.21.15.42:0/4181738786 <== mon.2 v2:172.21.15.111:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff5b009a800 con 0x55acf7728d40 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff5cdaf7b80 10 client.25703 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff5cdaf7b80 10 client.25703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff5cdaf7b80 20 client.25703 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55acf773c9d0 0x55acf773edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff5cdaf7b80 4 client.25703 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.617+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/4181738786 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acf76f2ee0 con 0x55acf773c9d0 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.618+0000 7ff5baffd700 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55acf773c9d0 0x55acf773edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.618+0000 7ff5baffd700 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55acf773c9d0 0x55acf773edf0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc0bcde705fe6be server_cookie=616d073ef43e56e in_seq=0 out_seq=0 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.618+0000 7ff59affd700 10 client.25703 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff59affd700 1 -- 172.21.15.42:0/4181738786 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acf76f2ee0 con 0x55acf773c9d0 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff59affd700 10 client.25703 handle_command_reply: tid=0 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 client.25703 shutdown 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/4181738786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7728d40 msgr2=0x55acf772c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7728d40 0x55acf772c930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005f50 tx=0x7ff5b0005c70 comp rx=0 tx=0).stop 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/4181738786 shutdown_connections 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 0x55acf772d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7723ac0 0x55acf772f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55acf773c9d0 0x55acf773edf0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/4181738786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7728d40 0x55acf772c930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff59affd700 0 client.25703 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/4181738786 >> 172.21.15.42:0/4181738786 conn(0x55acf7652780 msgr2=0x55acf7722570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/4181738786 shutdown_connections 2023-12-12T10:04:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/4181738786 wait complete. 2023-12-12T10:04:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.619+0000 7ff5cdaf7b80 20 client.25703 trim_cache size 0 max 16384 2023-12-12T10:04:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5cdaf7b80 1 Processor -- start 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5cdaf7b80 1 -- start start 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7743710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7733d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 0x55acf7734270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf772d600 con 0x55acf7728d40 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf77294b0 con 0x55acf7723ac0 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf771a910 con 0x55acf7729660 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7733d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5baffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7743710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7733d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36884/0 (socket says 172.21.15.42:36884) 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/3572839925 learned_addr learned my addr 172.21.15.42:0/3572839925 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.639+0000 7ff5bb7fe700 1 --2- 172.21.15.42:0/3572839925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 0x55acf7734270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/3572839925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 msgr2=0x55acf7734270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/3572839925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 0x55acf7734270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/3572839925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 msgr2=0x55acf7743710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/3572839925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7743710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/3572839925 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf74a0d80 con 0x55acf7728d40 2023-12-12T10:04:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5baffd700 1 --2- 172.21.15.42:0/3572839925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7743710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/3572839925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7733d30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0097d80 tx=0x7ff5b00998c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7da7e1833b26a7c server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5a27fc700 1 -- 172.21.15.42:0/3572839925 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0097650 con 0x55acf7728d40 2023-12-12T10:04:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5a27fc700 1 -- 172.21.15.42:0/3572839925 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55acf7716730 con 0x55acf7728d40 2023-12-12T10:04:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.640+0000 7ff5a27fc700 1 -- 172.21.15.42:0/3572839925 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00a32d0 con 0x55acf7728d40 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/3572839925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 msgr2=0x55acf7733d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/3572839925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7733d30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0097d80 tx=0x7ff5b00998c0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/3572839925 shutdown_connections 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/3572839925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7723ac0 0x55acf7743710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/3572839925 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7729660 0x55acf7734270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/3572839925 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7733d30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/3572839925 >> 172.21.15.42:0/3572839925 conn(0x55acf7652780 msgr2=0x55acf771f530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/3572839925 shutdown_connections 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/3572839925 wait complete. 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.641+0000 7ff5cdaf7b80 1 -- start start 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5cdaf7b80 1 Processor -- start 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7723ac0 0x55acf7734210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7731d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5cdaf7b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7729660 0x55acf7732430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf7732b70 con 0x55acf7728d40 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf7791e90 con 0x55acf7729660 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5cdaf7b80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf7791fd0 con 0x55acf7723ac0 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5cdaf7b80 10 client.0 _ll_register_callbacks cb 0x55acf755ca20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7731d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5ba7fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7731d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36890/0 (socket says 172.21.15.42:36890) 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/2988580197 learned_addr learned my addr 172.21.15.42:0/2988580197 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5bb7fe700 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7729660 0x55acf7732430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5baffd700 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7723ac0 0x55acf7734210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/2988580197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7723ac0 msgr2=0x55acf7734210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7723ac0 0x55acf7734210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/2988580197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7729660 msgr2=0x55acf7732430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7729660 0x55acf7732430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5ba7fc700 1 -- 172.21.15.42:0/2988580197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ac017070 con 0x55acf7728d40 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.642+0000 7ff5bb7fe700 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7729660 0x55acf7732430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5ba7fc700 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7731d10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005f50 tx=0x7ff5b0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2524b540055ccde6 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff59affd700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff59affd700 1 -- 172.21.15.42:0/2988580197 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b009f460 con 0x55acf7728d40 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff59affd700 1 -- 172.21.15.42:0/2988580197 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff5b00a8060 con 0x55acf7728d40 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/2988580197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5b00cd070 con 0x55acf7728d40 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff59affd700 1 -- 172.21.15.42:0/2988580197 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5b009b310 con 0x55acf7728d40 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5cdaf7b80 10 client.16014 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/2988580197 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7ff5bc1a67e0 con 0x55acf7728d40 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff59affd700 1 -- 172.21.15.42:0/2988580197 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5b00cd070 con 0x55acf7728d40 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff59affd700 1 -- 172.21.15.42:0/2988580197 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7ff5b00b92c0 con 0x55acf7728d40 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5cdaf7b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5cdaf7b80 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5cdaf7b80 20 client.16014 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55acf77932b0 0x55acf77956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5cdaf7b80 4 client.16014 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:04:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.643+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/2988580197 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55acf763df70 con 0x55acf77932b0 2023-12-12T10:04:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.644+0000 7ff5baffd700 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55acf77932b0 0x55acf77956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.644+0000 7ff59affd700 10 client.16014 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.644+0000 7ff5baffd700 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55acf77932b0 0x55acf77956d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0e3c7cdb123302 server_cookie=f54651e76240f125 in_seq=0 out_seq=0 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff59affd700 1 -- 172.21.15.42:0/2988580197 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55acf763df70 con 0x55acf77932b0 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff59affd700 10 client.16014 handle_command_reply: tid=0 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 client.16014 shutdown 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/2988580197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 msgr2=0x55acf7731d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7731d10 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0005f50 tx=0x7ff5b0005c00 comp rx=0 tx=0).stop 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/2988580197 shutdown_connections 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55acf77932b0 0x55acf77956d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55acf7729660 0x55acf7732430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff59affd700 0 client.16014 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55acf7723ac0 0x55acf7734210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 --2- 172.21.15.42:0/2988580197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acf7728d40 0x55acf7731d10 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/2988580197 >> 172.21.15.42:0/2988580197 conn(0x55acf7652780 msgr2=0x55acf7743710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/2988580197 shutdown_connections 2023-12-12T10:04:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 1 -- 172.21.15.42:0/2988580197 wait complete. 2023-12-12T10:04:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.655+0000 7ff5cdaf7b80 20 client.16014 trim_cache size 0 max 16384 2023-12-12T10:04:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:04:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:04:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:04:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:04:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 -- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7ff5a40a15f0 msgr2=0x7ff5a40a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7ff5a40a15f0 0x7ff5a40a3ab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009be0 tx=0x7ff5ac009260 comp rx=0 tx=0).stop 2023-12-12T10:04:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 -- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc100d90 msgr2=0x7ff5bc1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc100d90 0x7ff5bc1025b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0000c00 tx=0x7ff5b0004730 comp rx=0 tx=0).stop 2023-12-12T10:04:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 -- 172.21.15.42:0/577427459 shutdown_connections 2023-12-12T10:04:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7ff5a40a15f0 0x7ff5a40a3ab0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff5bc106480 0x7ff5bc102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc1003d0 0x7ff5bc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 --2- 172.21.15.42:0/577427459 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5bc100d90 0x7ff5bc1025b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 -- 172.21.15.42:0/577427459 >> 172.21.15.42:0/577427459 conn(0x7ff5bc077570 msgr2=0x7ff5bc068ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 -- 172.21.15.42:0/577427459 shutdown_connections 2023-12-12T10:04:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:27.656+0000 7ff5c18fe700 1 -- 172.21.15.42:0/577427459 wait complete. 2023-12-12T10:04:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:28 smithi173 ceph-mon[103308]: pgmap v178: 129 pgs: 129 active+clean; 652 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 395 op/s 2023-12-12T10:04:28.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:28 smithi042 ceph-mon[98894]: pgmap v178: 129 pgs: 129 active+clean; 652 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 395 op/s 2023-12-12T10:04:28.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:28 smithi111 ceph-mon[102565]: pgmap v178: 129 pgs: 129 active+clean; 652 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.1 MiB/s wr, 395 op/s 2023-12-12T10:04:30.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:30 smithi173 ceph-mon[103308]: pgmap v179: 129 pgs: 129 active+clean; 693 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.6 MiB/s wr, 564 op/s 2023-12-12T10:04:30.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:30 smithi042 ceph-mon[98894]: pgmap v179: 129 pgs: 129 active+clean; 693 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.6 MiB/s wr, 564 op/s 2023-12-12T10:04:30.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:30 smithi111 ceph-mon[102565]: pgmap v179: 129 pgs: 129 active+clean; 693 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.6 MiB/s wr, 564 op/s 2023-12-12T10:04:32.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:32 smithi173 ceph-mon[103308]: pgmap v180: 129 pgs: 129 active+clean; 693 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 308 op/s 2023-12-12T10:04:32.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:32 smithi042 ceph-mon[98894]: pgmap v180: 129 pgs: 129 active+clean; 693 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 308 op/s 2023-12-12T10:04:32.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:32 smithi111 ceph-mon[102565]: pgmap v180: 129 pgs: 129 active+clean; 693 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 6.4 MiB/s wr, 308 op/s 2023-12-12T10:04:34.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:34 smithi173 ceph-mon[103308]: pgmap v181: 129 pgs: 129 active+clean; 697 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.7 MiB/s wr, 310 op/s 2023-12-12T10:04:34.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:34 smithi042 ceph-mon[98894]: pgmap v181: 129 pgs: 129 active+clean; 697 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.7 MiB/s wr, 310 op/s 2023-12-12T10:04:34.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:34 smithi111 ceph-mon[102565]: pgmap v181: 129 pgs: 129 active+clean; 697 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.7 MiB/s wr, 310 op/s 2023-12-12T10:04:36.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:36 smithi173 ceph-mon[103308]: pgmap v182: 129 pgs: 129 active+clean; 715 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.9 MiB/s wr, 371 op/s 2023-12-12T10:04:36.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:36 smithi042 ceph-mon[98894]: pgmap v182: 129 pgs: 129 active+clean; 715 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.9 MiB/s wr, 371 op/s 2023-12-12T10:04:36.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:36 smithi111 ceph-mon[102565]: pgmap v182: 129 pgs: 129 active+clean; 715 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.9 MiB/s wr, 371 op/s 2023-12-12T10:04:38.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:38 smithi173 ceph-mon[103308]: pgmap v183: 129 pgs: 129 active+clean; 715 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 246 op/s 2023-12-12T10:04:38.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:38 smithi042 ceph-mon[98894]: pgmap v183: 129 pgs: 129 active+clean; 715 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 246 op/s 2023-12-12T10:04:38.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:38 smithi111 ceph-mon[102565]: pgmap v183: 129 pgs: 129 active+clean; 715 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.3 MiB/s wr, 246 op/s 2023-12-12T10:04:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.777+0000 7f42b4366700 1 -- 172.21.15.42:0/319907956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac100d90 msgr2=0x7f42ac10d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.777+0000 7f42b4366700 1 --2- 172.21.15.42:0/319907956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac100d90 0x7f42ac10d2a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f42a400b0a0 tx=0x7f42a400b3b0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.777+0000 7f42b4366700 1 -- 172.21.15.42:0/319907956 shutdown_connections 2023-12-12T10:04:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.777+0000 7f42b4366700 1 --2- 172.21.15.42:0/319907956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ac106470 0x7f42ac106850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.777+0000 7f42b4366700 1 --2- 172.21.15.42:0/319907956 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac100d90 0x7f42ac10d2a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.777+0000 7f42b4366700 1 --2- 172.21.15.42:0/319907956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ac1003d0 0x7f42ac100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.777+0000 7f42b4366700 1 -- 172.21.15.42:0/319907956 >> 172.21.15.42:0/319907956 conn(0x7f42ac0fbea0 msgr2=0x7f42ac0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.777+0000 7f42b4366700 1 -- 172.21.15.42:0/319907956 shutdown_connections 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.777+0000 7f42b4366700 1 -- 172.21.15.42:0/319907956 wait complete. 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b4366700 1 Processor -- start 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b4366700 1 -- start start 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b4366700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ac1003d0 0x7f42ac1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b4366700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ac100d90 0x7f42ac198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b4366700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac106470 0x7f42ac1a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b4366700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1994f0 con 0x7f42ac106470 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b4366700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1a3c80 con 0x7f42ac1003d0 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b4366700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ac1a3df0 con 0x7f42ac100d90 2023-12-12T10:04:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b2903700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac106470 0x7f42ac1a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b1901700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ac100d90 0x7f42ac198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b2903700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac106470 0x7f42ac1a3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36906/0 (socket says 172.21.15.42:36906) 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.778+0000 7f42b2903700 1 -- 172.21.15.42:0/2468293328 learned_addr learned my addr 172.21.15.42:0/2468293328 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42b2102700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ac1003d0 0x7f42ac1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42b2903700 1 -- 172.21.15.42:0/2468293328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ac100d90 msgr2=0x7f42ac198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42b2903700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ac100d90 0x7f42ac198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42b2903700 1 -- 172.21.15.42:0/2468293328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ac1003d0 msgr2=0x7f42ac1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42b2903700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ac1003d0 0x7f42ac1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42b2903700 1 -- 172.21.15.42:0/2468293328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a4009d00 con 0x7f42ac106470 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42b2903700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac106470 0x7f42ac1a3740 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f42a400b070 tx=0x7f42a4008ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42a37fe700 1 -- 172.21.15.42:0/2468293328 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a400e040 con 0x7f42ac106470 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42a37fe700 1 -- 172.21.15.42:0/2468293328 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42a4003c00 con 0x7f42ac106470 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42b4366700 1 -- 172.21.15.42:0/2468293328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ac1a4070 con 0x7f42ac106470 2023-12-12T10:04:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42a37fe700 1 -- 172.21.15.42:0/2468293328 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40052a0 con 0x7f42ac106470 2023-12-12T10:04:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.779+0000 7f42b4366700 1 -- 172.21.15.42:0/2468293328 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac1a44f0 con 0x7f42ac106470 2023-12-12T10:04:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.782+0000 7f42a37fe700 1 -- 172.21.15.42:0/2468293328 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f42a401d470 con 0x7f42ac106470 2023-12-12T10:04:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.783+0000 7f42a37fe700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f42980a15f0 0x7f42980a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.783+0000 7f42b2102700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f42980a15f0 0x7f42980a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.783+0000 7f42a37fe700 1 -- 172.21.15.42:0/2468293328 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f42a40ce7e0 con 0x7f42ac106470 2023-12-12T10:04:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.783+0000 7f42b2102700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f42980a15f0 0x7f42980a3ab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f429c005950 tx=0x7f429c0058e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.783+0000 7f42c055fb80 1 Processor -- start 2023-12-12T10:04:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42c055fb80 1 -- start start 2023-12-12T10:04:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab6d40 0x562b4eab7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eab1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab1ac0 0x562b4eab1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42c055fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4ea0caf0 con 0x562b4eab1ac0 2023-12-12T10:04:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42c055fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4ea0a410 con 0x562b4eab6d40 2023-12-12T10:04:40.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42c055fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4ea05d00 con 0x562b4eab7660 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2903700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab1ac0 0x562b4eab1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2903700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab1ac0 0x562b4eab1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36914/0 (socket says 172.21.15.42:36914) 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2903700 1 -- 172.21.15.42:0/3006057075 learned_addr learned my addr 172.21.15.42:0/3006057075 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b1901700 1 --2- 172.21.15.42:0/3006057075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eab1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2102700 1 --2- 172.21.15.42:0/3006057075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab6d40 0x562b4eab7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2903700 1 -- 172.21.15.42:0/3006057075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 msgr2=0x562b4eab1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2903700 1 --2- 172.21.15.42:0/3006057075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eab1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2903700 1 -- 172.21.15.42:0/3006057075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab6d40 msgr2=0x562b4eab7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2903700 1 --2- 172.21.15.42:0/3006057075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab6d40 0x562b4eab7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2903700 1 -- 172.21.15.42:0/3006057075 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b4e8825d0 con 0x562b4eab1ac0 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.784+0000 7f42b2903700 1 --2- 172.21.15.42:0/3006057075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab1ac0 0x562b4eab1f00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f42a40a3270 tx=0x7f42a401dbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9e7955e7e6d75d server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42a0d82700 1 -- 172.21.15.42:0/3006057075 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a409bba0 con 0x562b4eab1ac0 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42a0d82700 1 -- 172.21.15.42:0/3006057075 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42a409b4a0 con 0x562b4eab1ac0 2023-12-12T10:04:40.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42a0d82700 1 -- 172.21.15.42:0/3006057075 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40ce410 con 0x562b4eab1ac0 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42c055fb80 1 -- 172.21.15.42:0/3006057075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab1ac0 msgr2=0x562b4eab1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3006057075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab1ac0 0x562b4eab1f00 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f42a40a3270 tx=0x7f42a401dbf0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42c055fb80 1 -- 172.21.15.42:0/3006057075 shutdown_connections 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3006057075 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab6d40 0x562b4eab7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3006057075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab1ac0 0x562b4eab1f00 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3006057075 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eab1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.785+0000 7f42c055fb80 1 -- 172.21.15.42:0/3006057075 >> 172.21.15.42:0/3006057075 conn(0x562b4e9e0780 msgr2=0x562b4eaabc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 -- 172.21.15.42:0/3006057075 shutdown_connections 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 -- 172.21.15.42:0/3006057075 wait complete. 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 -- start start 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 Processor -- start 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eabaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eabaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eabb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4e9bf500 con 0x562b4eab6d40 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4e9bf1a0 con 0x562b4eab1ac0 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4e8848c0 con 0x562b4eab7660 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42c055fb80 10 client.0 _ll_register_callbacks cb 0x562b4e8eaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42b1901700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eabaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42b1901700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eabaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36918/0 (socket says 172.21.15.42:36918) 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.786+0000 7f42b1901700 1 -- 172.21.15.42:0/3060632777 learned_addr learned my addr 172.21.15.42:0/3060632777 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42b2903700 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eabb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42b2102700 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eabaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42b1901700 1 -- 172.21.15.42:0/3060632777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 msgr2=0x562b4eabb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42b1901700 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eabb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42b1901700 1 -- 172.21.15.42:0/3060632777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 msgr2=0x562b4eabaa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:40.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42b1901700 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eabaa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42b1901700 1 -- 172.21.15.42:0/3060632777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ac1a44f0 con 0x562b4eab6d40 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42b1901700 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eabaf50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f42a8012770 tx=0x7f42a8012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8020b975485ec9ca server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f4295ffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f4295ffb700 1 -- 172.21.15.42:0/3060632777 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8017070 con 0x562b4eab6d40 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f4295ffb700 1 -- 172.21.15.42:0/3060632777 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f42a800f480 con 0x562b4eab6d40 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42c055fb80 1 -- 172.21.15.42:0/3060632777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b4eaa4730 con 0x562b4eab6d40 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f4295ffb700 1 -- 172.21.15.42:0/3060632777 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42a800f640 con 0x562b4eab6d40 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42c055fb80 10 client.16026 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.787+0000 7f42c055fb80 1 -- 172.21.15.42:0/3060632777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562b4e82ed80 con 0x562b4eab6d40 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.788+0000 7f4295ffb700 1 -- 172.21.15.42:0/3060632777 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562b4eaa4730 con 0x562b4eab6d40 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.788+0000 7f4295ffb700 1 -- 172.21.15.42:0/3060632777 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f42a8010830 con 0x562b4eab6d40 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.788+0000 7f42c055fb80 10 client.16026 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.788+0000 7f42c055fb80 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.788+0000 7f42c055fb80 20 client.16026 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.788+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562b4eacaf10 0x562b4eacd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.788+0000 7f42c055fb80 4 client.16026 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:04:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.788+0000 7f42c055fb80 1 -- 172.21.15.42:0/3060632777 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b4ea80ee0 con 0x562b4eacaf10 2023-12-12T10:04:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.788+0000 7f42b2102700 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562b4eacaf10 0x562b4eacd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42b2102700 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562b4eacaf10 0x562b4eacd330 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241f888193f86249 server_cookie=11666116eb472926 in_seq=0 out_seq=0 2023-12-12T10:04:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f4295ffb700 10 client.16026 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f4295ffb700 1 -- 172.21.15.42:0/3060632777 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b4ea80ee0 con 0x562b4eacaf10 2023-12-12T10:04:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f4295ffb700 10 client.16026 handle_command_reply: tid=0 2023-12-12T10:04:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42c055fb80 1 client.16026 shutdown 2023-12-12T10:04:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42c055fb80 1 -- 172.21.15.42:0/3060632777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 msgr2=0x562b4eabaf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eabaf50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f42a8012770 tx=0x7f42a8012a80 comp rx=0 tx=0).stop 2023-12-12T10:04:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42c055fb80 1 -- 172.21.15.42:0/3060632777 shutdown_connections 2023-12-12T10:04:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562b4eacaf10 0x562b4eacd330 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eabaa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eabb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f4295ffb700 0 client.16026 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42c055fb80 1 --2- 172.21.15.42:0/3060632777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eabaf50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.789+0000 7f42c055fb80 1 -- 172.21.15.42:0/3060632777 >> 172.21.15.42:0/3060632777 conn(0x562b4e9e0780 msgr2=0x562b4eab0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.790+0000 7f42c055fb80 1 -- 172.21.15.42:0/3060632777 shutdown_connections 2023-12-12T10:04:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.790+0000 7f42c055fb80 1 -- 172.21.15.42:0/3060632777 wait complete. 2023-12-12T10:04:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.790+0000 7f42c055fb80 20 client.16026 trim_cache size 0 max 16384 2023-12-12T10:04:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42c055fb80 1 Processor -- start 2023-12-12T10:04:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42c055fb80 1 -- start start 2023-12-12T10:04:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eac1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eac2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eab4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42c055fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4eabbfc0 con 0x562b4eab6d40 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42c055fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4eab22b0 con 0x562b4eab1ac0 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42c055fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4eaa4700 con 0x562b4eab7660 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42b1901700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eac2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42b1901700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eac2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36926/0 (socket says 172.21.15.42:36926) 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42b2903700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eab4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42b1901700 1 -- 172.21.15.42:0/1161620524 learned_addr learned my addr 172.21.15.42:0/1161620524 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.805+0000 7f42b2102700 1 --2- 172.21.15.42:0/1161620524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eac1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42b1901700 1 -- 172.21.15.42:0/1161620524 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 msgr2=0x562b4eab4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42b1901700 1 --2- 172.21.15.42:0/1161620524 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eab4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42b1901700 1 -- 172.21.15.42:0/1161620524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 msgr2=0x562b4eac1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42b1901700 1 --2- 172.21.15.42:0/1161620524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eac1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42b1901700 1 -- 172.21.15.42:0/1161620524 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b4e817c00 con 0x562b4eab6d40 2023-12-12T10:04:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42b1901700 1 --2- 172.21.15.42:0/1161620524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eac2300 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f42a8010c10 tx=0x7f42a8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93245e2824e8a40 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42a1583700 1 -- 172.21.15.42:0/1161620524 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8017040 con 0x562b4eab6d40 2023-12-12T10:04:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42a1583700 1 -- 172.21.15.42:0/1161620524 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42a80057a0 con 0x562b4eab6d40 2023-12-12T10:04:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42a1583700 1 -- 172.21.15.42:0/1161620524 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a800fd30 con 0x562b4eab6d40 2023-12-12T10:04:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42c055fb80 1 -- 172.21.15.42:0/1161620524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 msgr2=0x562b4eac2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42c055fb80 1 --2- 172.21.15.42:0/1161620524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eac2300 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f42a8010c10 tx=0x7f42a8009750 comp rx=0 tx=0).stop 2023-12-12T10:04:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42c055fb80 1 -- 172.21.15.42:0/1161620524 shutdown_connections 2023-12-12T10:04:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42c055fb80 1 --2- 172.21.15.42:0/1161620524 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eac1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42c055fb80 1 --2- 172.21.15.42:0/1161620524 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eab4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42c055fb80 1 --2- 172.21.15.42:0/1161620524 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eac2300 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.806+0000 7f42c055fb80 1 -- 172.21.15.42:0/1161620524 >> 172.21.15.42:0/1161620524 conn(0x562b4e9e0780 msgr2=0x562b4eaabbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 -- 172.21.15.42:0/1161620524 shutdown_connections 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 -- 172.21.15.42:0/1161620524 wait complete. 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 -- start start 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 Processor -- start 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eb15550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eb15a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eb1a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4eb1a8f0 con 0x562b4eab6d40 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4eb1aa60 con 0x562b4eab1ac0 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b4eb1ac20 con 0x562b4eab7660 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42c055fb80 10 client.0 _ll_register_callbacks cb 0x562b4e8eaa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42b1901700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eb15a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42b1901700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eb15a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36932/0 (socket says 172.21.15.42:36932) 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42b2102700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eb15550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42b1901700 1 -- 172.21.15.42:0/354515286 learned_addr learned my addr 172.21.15.42:0/354515286 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.807+0000 7f42b2903700 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eb1a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42b1901700 1 -- 172.21.15.42:0/354515286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 msgr2=0x562b4eb1a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42b1901700 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eb1a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42b1901700 1 -- 172.21.15.42:0/354515286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 msgr2=0x562b4eb15550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42b1901700 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eb15550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42b1901700 1 -- 172.21.15.42:0/354515286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b4e82ed80 con 0x562b4eab6d40 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42b1901700 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eb15a90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f42a8010b90 tx=0x7f42a800f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c960bc64b7d8f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f4295ffb700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f4295ffb700 1 -- 172.21.15.42:0/354515286 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8017040 con 0x562b4eab6d40 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f4295ffb700 1 -- 172.21.15.42:0/354515286 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f42a8000e40 con 0x562b4eab6d40 2023-12-12T10:04:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42c055fb80 1 -- 172.21.15.42:0/354515286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42a409b4a0 con 0x562b4eab6d40 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f4295ffb700 1 -- 172.21.15.42:0/354515286 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42a8005240 con 0x562b4eab6d40 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42c055fb80 10 client.16032 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42c055fb80 1 -- 172.21.15.42:0/354515286 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x562b4e8551f0 con 0x562b4eab6d40 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f4295ffb700 1 -- 172.21.15.42:0/354515286 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42a409b4a0 con 0x562b4eab6d40 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f4295ffb700 1 -- 172.21.15.42:0/354515286 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7f42a800f880 con 0x562b4eab6d40 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42c055fb80 10 client.16032 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42c055fb80 10 client.16032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.808+0000 7f42c055fb80 20 client.16032 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.809+0000 7f42c055fb80 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562b4eb1bfc0 0x562b4eb28820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.809+0000 7f42c055fb80 4 client.16032 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.809+0000 7f42c055fb80 1 -- 172.21.15.42:0/354515286 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b4e9cbf70 con 0x562b4eb1bfc0 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.809+0000 7f42b2102700 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562b4eb1bfc0 0x562b4eb28820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.809+0000 7f42b2102700 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562b4eb1bfc0 0x562b4eb28820 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13d99c0897bfc9f8 server_cookie=e229c01fbd6561ba in_seq=0 out_seq=0 2023-12-12T10:04:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.809+0000 7f4295ffb700 10 client.16032 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f4295ffb700 1 -- 172.21.15.42:0/354515286 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b4e9cbf70 con 0x562b4eb1bfc0 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f4295ffb700 10 client.16032 handle_command_reply: tid=0 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 client.16032 shutdown 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 -- 172.21.15.42:0/354515286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 msgr2=0x562b4eb15a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eb15a90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f42a8010b90 tx=0x7f42a800f480 comp rx=0 tx=0).stop 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 -- 172.21.15.42:0/354515286 shutdown_connections 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x562b4eb1bfc0 0x562b4eb28820 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x562b4eab1ac0 0x562b4eb15550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f4295ffb700 0 client.16032 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x562b4eab7660 0x562b4eb1a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 --2- 172.21.15.42:0/354515286 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x562b4eab6d40 0x562b4eb15a90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 -- 172.21.15.42:0/354515286 >> 172.21.15.42:0/354515286 conn(0x562b4e9e0780 msgr2=0x562b4eab4880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.814+0000 7f42c055fb80 1 -- 172.21.15.42:0/354515286 shutdown_connections 2023-12-12T10:04:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42c055fb80 1 -- 172.21.15.42:0/354515286 wait complete. 2023-12-12T10:04:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42c055fb80 20 client.16032 trim_cache size 0 max 16384 2023-12-12T10:04:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:04:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:04:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:04:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:04:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 -- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f42980a15f0 msgr2=0x7f42980a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f42980a15f0 0x7f42980a3ab0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f429c005950 tx=0x7f429c0058e0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 -- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac106470 msgr2=0x7f42ac1a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac106470 0x7f42ac1a3740 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f42a400b070 tx=0x7f42a4008ff0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 -- 172.21.15.42:0/2468293328 shutdown_connections 2023-12-12T10:04:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f42980a15f0 0x7f42980a3ab0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42ac1003d0 0x7f42ac1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f42ac106470 0x7f42ac1a3740 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 --2- 172.21.15.42:0/2468293328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ac100d90 0x7f42ac198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 -- 172.21.15.42:0/2468293328 >> 172.21.15.42:0/2468293328 conn(0x7f42ac0fbea0 msgr2=0x7f42ac0fc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 -- 172.21.15.42:0/2468293328 shutdown_connections 2023-12-12T10:04:40.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:40.815+0000 7f42b4366700 1 -- 172.21.15.42:0/2468293328 wait complete. 2023-12-12T10:04:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:40 smithi173 ceph-mon[103308]: pgmap v184: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 263 op/s 2023-12-12T10:04:40.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:40 smithi042 ceph-mon[98894]: pgmap v184: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 263 op/s 2023-12-12T10:04:40.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:40 smithi111 ceph-mon[102565]: pgmap v184: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.4 MiB/s wr, 263 op/s 2023-12-12T10:04:41.407 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-12T10:04:41.407 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-12T10:04:41.487 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2023-12-12T10:04:41.488 DEBUG:teuthology.orchestra.run.smithi042:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2023-12-12T10:04:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:42 smithi173 ceph-mon[103308]: pgmap v185: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 94 op/s 2023-12-12T10:04:42.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:42 smithi042 ceph-mon[98894]: pgmap v185: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 94 op/s 2023-12-12T10:04:42.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:42 smithi111 ceph-mon[102565]: pgmap v185: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 94 op/s 2023-12-12T10:04:44.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:44 smithi042 ceph-mon[98894]: pgmap v186: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 105 op/s 2023-12-12T10:04:44.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:44 smithi111 ceph-mon[102565]: pgmap v186: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 105 op/s 2023-12-12T10:04:45.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:44 smithi173 ceph-mon[103308]: pgmap v186: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 105 op/s 2023-12-12T10:04:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:46 smithi042 ceph-mon[98894]: pgmap v187: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2023-12-12T10:04:46.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:46 smithi111 ceph-mon[102565]: pgmap v187: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2023-12-12T10:04:47.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:46 smithi173 ceph-mon[103308]: pgmap v187: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.6 MiB/s wr, 148 op/s 2023-12-12T10:04:48.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:48 smithi042 ceph-mon[98894]: pgmap v188: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 139 KiB/s wr, 71 op/s 2023-12-12T10:04:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:48 smithi111 ceph-mon[102565]: pgmap v188: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 139 KiB/s wr, 71 op/s 2023-12-12T10:04:49.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:48 smithi173 ceph-mon[103308]: pgmap v188: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 139 KiB/s wr, 71 op/s 2023-12-12T10:04:50.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:50 smithi042 ceph-mon[98894]: pgmap v189: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 157 KiB/s wr, 79 op/s 2023-12-12T10:04:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:50 smithi111 ceph-mon[102565]: pgmap v189: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 157 KiB/s wr, 79 op/s 2023-12-12T10:04:51.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:50 smithi173 ceph-mon[103308]: pgmap v189: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 343 KiB/s rd, 157 KiB/s wr, 79 op/s 2023-12-12T10:04:51.598 INFO:tasks.workunit.client.0.smithi042.stdout:ok 2023-12-12T10:04:51.601 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-12T10:04:51.601 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-12T10:04:51.692 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2023-12-12T10:04:51.692 DEBUG:teuthology.orchestra.run.smithi042:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2023-12-12T10:04:52.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:52 smithi042 ceph-mon[98894]: pgmap v190: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 79 KiB/s wr, 62 op/s 2023-12-12T10:04:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:52 smithi111 ceph-mon[102565]: pgmap v190: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 79 KiB/s wr, 62 op/s 2023-12-12T10:04:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:52 smithi173 ceph-mon[103308]: pgmap v190: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 79 KiB/s wr, 62 op/s 2023-12-12T10:04:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.940+0000 7fcec9294700 1 -- 172.21.15.42:0/1861272513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 msgr2=0x7fcec4106850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.940+0000 7fcec9294700 1 --2- 172.21.15.42:0/1861272513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 0x7fcec4106850 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fceac009990 tx=0x7fceac009ca0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.940+0000 7fcec9294700 1 -- 172.21.15.42:0/1861272513 shutdown_connections 2023-12-12T10:04:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.940+0000 7fcec9294700 1 --2- 172.21.15.42:0/1861272513 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcec4100d90 0x7fcec410d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.940+0000 7fcec9294700 1 --2- 172.21.15.42:0/1861272513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 0x7fcec4106850 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.940+0000 7fcec9294700 1 --2- 172.21.15.42:0/1861272513 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcec41003d0 0x7fcec4100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.940+0000 7fcec9294700 1 -- 172.21.15.42:0/1861272513 >> 172.21.15.42:0/1861272513 conn(0x7fcec40fbea0 msgr2=0x7fcec40fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.940+0000 7fcec9294700 1 -- 172.21.15.42:0/1861272513 shutdown_connections 2023-12-12T10:04:53.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.940+0000 7fcec9294700 1 -- 172.21.15.42:0/1861272513 wait complete. 2023-12-12T10:04:53.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec9294700 1 Processor -- start 2023-12-12T10:04:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec9294700 1 -- start start 2023-12-12T10:04:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec9294700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcec41003d0 0x7fcec4198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec9294700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcec4100d90 0x7fcec4198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec9294700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 0x7fcec4199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec9294700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec41a3a80 con 0x7fcec4106470 2023-12-12T10:04:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec9294700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec41a3bf0 con 0x7fcec41003d0 2023-12-12T10:04:53.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec9294700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec41a3db0 con 0x7fcec4100d90 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 0x7fcec4199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 0x7fcec4199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36948/0 (socket says 172.21.15.42:36948) 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec37fe700 1 -- 172.21.15.42:0/3412108672 learned_addr learned my addr 172.21.15.42:0/3412108672 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec27fc700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcec4100d90 0x7fcec4198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.941+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcec41003d0 0x7fcec4198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcec37fe700 1 -- 172.21.15.42:0/3412108672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcec4100d90 msgr2=0x7fcec4198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcec37fe700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcec4100d90 0x7fcec4198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcec37fe700 1 -- 172.21.15.42:0/3412108672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcec41003d0 msgr2=0x7fcec4198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcec37fe700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcec41003d0 0x7fcec4198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcec37fe700 1 -- 172.21.15.42:0/3412108672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac0095f0 con 0x7fcec4106470 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcec37fe700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 0x7fcec4199230 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcebc007ee0 tx=0x7fcebc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcebbfff700 1 -- 172.21.15.42:0/3412108672 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc017070 con 0x7fcec4106470 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcebbfff700 1 -- 172.21.15.42:0/3412108672 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcebc00f040 con 0x7fcec4106470 2023-12-12T10:04:53.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcec9294700 1 -- 172.21.15.42:0/3412108672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec41a4090 con 0x7fcec4106470 2023-12-12T10:04:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcebbfff700 1 -- 172.21.15.42:0/3412108672 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc005740 con 0x7fcec4106470 2023-12-12T10:04:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.942+0000 7fcec9294700 1 -- 172.21.15.42:0/3412108672 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec41a4550 con 0x7fcec4106470 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.945+0000 7fcebbfff700 1 -- 172.21.15.42:0/3412108672 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fcebc00dc60 con 0x7fcec4106470 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.945+0000 7fcebbfff700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fceb00a15f0 0x7fceb00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.945+0000 7fcebbfff700 1 -- 172.21.15.42:0/3412108672 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcebc0ccc40 con 0x7fcec4106470 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.945+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fceb00a15f0 0x7fceb00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.946+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fceb00a15f0 0x7fceb00a3ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fceac006010 tx=0x7fceac00b260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.946+0000 7fced548db80 1 Processor -- start 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.946+0000 7fced548db80 1 -- start start 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.946+0000 7fced548db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 0x55f1769aff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.946+0000 7fced548db80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 0x55f1769b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.946+0000 7fced548db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.946+0000 7fced548db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f176906af0 con 0x55f1769b42d0 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.946+0000 7fced548db80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f176904410 con 0x55f1769b04a0 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.946+0000 7fced548db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1768ffd00 con 0x55f1769afb80 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769b4730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36956/0 (socket says 172.21.15.42:36956) 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec37fe700 1 -- 172.21.15.42:0/1171002702 learned_addr learned my addr 172.21.15.42:0/1171002702 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/1171002702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 0x55f1769aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec27fc700 1 --2- 172.21.15.42:0/1171002702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 0x55f1769b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec37fe700 1 -- 172.21.15.42:0/1171002702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 msgr2=0x55f1769aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec37fe700 1 --2- 172.21.15.42:0/1171002702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 0x55f1769aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec37fe700 1 -- 172.21.15.42:0/1171002702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 msgr2=0x55f1769b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec37fe700 1 --2- 172.21.15.42:0/1171002702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 0x55f1769b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec37fe700 1 -- 172.21.15.42:0/1171002702 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f17677c5d0 con 0x55f1769b42d0 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fcec37fe700 1 --2- 172.21.15.42:0/1171002702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769b4730 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcebc095dd0 tx=0x7fcebc00df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135ea74288f3b63e server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fceb97fa700 1 -- 172.21.15.42:0/1171002702 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc09da20 con 0x55f1769b42d0 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fceb97fa700 1 -- 172.21.15.42:0/1171002702 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcebc0a1c60 con 0x55f1769b42d0 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.947+0000 7fceb97fa700 1 -- 172.21.15.42:0/1171002702 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc005aa0 con 0x55f1769b42d0 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.948+0000 7fced548db80 1 -- 172.21.15.42:0/1171002702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 msgr2=0x55f1769b4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.948+0000 7fced548db80 1 --2- 172.21.15.42:0/1171002702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769b4730 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fcebc095dd0 tx=0x7fcebc00df60 comp rx=0 tx=0).stop 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.948+0000 7fced548db80 1 -- 172.21.15.42:0/1171002702 shutdown_connections 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.948+0000 7fced548db80 1 --2- 172.21.15.42:0/1171002702 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769b4730 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.948+0000 7fced548db80 1 --2- 172.21.15.42:0/1171002702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 0x55f1769aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.948+0000 7fced548db80 1 --2- 172.21.15.42:0/1171002702 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 0x55f1769b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.948+0000 7fced548db80 1 -- 172.21.15.42:0/1171002702 >> 172.21.15.42:0/1171002702 conn(0x55f1768da780 msgr2=0x55f1769a5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.948+0000 7fced548db80 1 -- 172.21.15.42:0/1171002702 shutdown_connections 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.948+0000 7fced548db80 1 -- 172.21.15.42:0/1171002702 wait complete. 2023-12-12T10:04:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fced548db80 1 -- start start 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fced548db80 1 Processor -- start 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fced548db80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 0x55f1769b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fced548db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fced548db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 0x55f1769b9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fced548db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1768b9500 con 0x55f1769b04a0 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fced548db80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1768b91a0 con 0x55f1769afb80 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fced548db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f17677e8c0 con 0x55f1769b42d0 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fced548db80 10 client.0 _ll_register_callbacks cb 0x55f1767e4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36960/0 (socket says 172.21.15.42:36960) 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec27fc700 1 -- 172.21.15.42:0/2963336876 learned_addr learned my addr 172.21.15.42:0/2963336876 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:53.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec37fe700 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 0x55f1769b9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 0x55f1769b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec27fc700 1 -- 172.21.15.42:0/2963336876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 msgr2=0x55f1769b9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec27fc700 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 0x55f1769b9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec27fc700 1 -- 172.21.15.42:0/2963336876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 msgr2=0x55f1769b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec27fc700 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 0x55f1769b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec27fc700 1 -- 172.21.15.42:0/2963336876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec41a4550 con 0x55f1769b04a0 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 0x55f1769b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.949+0000 7fcec37fe700 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 0x55f1769b9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fcec27fc700 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b93e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005c50 tx=0x7fceb4012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d384515fdbb737 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:53.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fceaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:04:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fceaa7fc700 1 -- 172.21.15.42:0/2963336876 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4017070 con 0x55f1769b04a0 2023-12-12T10:04:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fceaa7fc700 1 -- 172.21.15.42:0/2963336876 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fceb400f540 con 0x55f1769b04a0 2023-12-12T10:04:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fced548db80 1 -- 172.21.15.42:0/2963336876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f17699e730 con 0x55f1769b04a0 2023-12-12T10:04:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fceaa7fc700 1 -- 172.21.15.42:0/2963336876 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fceb40122e0 con 0x55f1769b04a0 2023-12-12T10:04:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fced548db80 10 client.16044 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fced548db80 1 -- 172.21.15.42:0/2963336876 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f176728d80 con 0x55f1769b04a0 2023-12-12T10:04:53.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fceaa7fc700 1 -- 172.21.15.42:0/2963336876 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f17699e730 con 0x55f1769b04a0 2023-12-12T10:04:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fceaa7fc700 1 -- 172.21.15.42:0/2963336876 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fceb40108b0 con 0x55f1769b04a0 2023-12-12T10:04:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fced548db80 10 client.16044 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fced548db80 10 client.16044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.950+0000 7fced548db80 20 client.16044 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:53.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.951+0000 7fced548db80 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f1769c96f0 0x55f1769cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.951+0000 7fced548db80 4 client.16044 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-12T10:04:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.951+0000 7fced548db80 1 -- 172.21.15.42:0/2963336876 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f17697aee0 con 0x55f1769c96f0 2023-12-12T10:04:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.951+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f1769c96f0 0x55f1769cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.951+0000 7fceaa7fc700 10 client.16044 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.951+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f1769c96f0 0x55f1769cbb10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4064ba1aa85aa3cf server_cookie=e7d3948b482fd11c in_seq=0 out_seq=0 2023-12-12T10:04:53.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fceaa7fc700 1 -- 172.21.15.42:0/2963336876 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f17697aee0 con 0x55f1769c96f0 2023-12-12T10:04:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fceaa7fc700 10 client.16044 handle_command_reply: tid=0 2023-12-12T10:04:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 client.16044 shutdown 2023-12-12T10:04:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 -- 172.21.15.42:0/2963336876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 msgr2=0x55f1769b93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b93e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fceb4005c50 tx=0x7fceb4012ad0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 -- 172.21.15.42:0/2963336876 shutdown_connections 2023-12-12T10:04:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f1769c96f0 0x55f1769cbb10 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 0x55f1769b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fceaa7fc700 0 client.16044 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 0x55f1769b9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 --2- 172.21.15.42:0/2963336876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b93e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 -- 172.21.15.42:0/2963336876 >> 172.21.15.42:0/2963336876 conn(0x55f1768da780 msgr2=0x55f1769aa570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 -- 172.21.15.42:0/2963336876 shutdown_connections 2023-12-12T10:04:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 1 -- 172.21.15.42:0/2963336876 wait complete. 2023-12-12T10:04:53.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.952+0000 7fced548db80 20 client.16044 trim_cache size 0 max 16384 2023-12-12T10:04:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fced548db80 1 Processor -- start 2023-12-12T10:04:53.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fced548db80 1 -- start start 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fced548db80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 0x55f1769b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fced548db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fced548db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 0x55f1769b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fced548db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f17699e550 con 0x55f1769b04a0 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fced548db80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1769ba4a0 con 0x55f1769afb80 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fced548db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f17699e700 con 0x55f1769b42d0 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec27fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36966/0 (socket says 172.21.15.42:36966) 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec27fc700 1 -- 172.21.15.42:0/3496797865 learned_addr learned my addr 172.21.15.42:0/3496797865 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec37fe700 1 --2- 172.21.15.42:0/3496797865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 0x55f1769b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/3496797865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 0x55f1769b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec27fc700 1 -- 172.21.15.42:0/3496797865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 msgr2=0x55f1769b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec27fc700 1 --2- 172.21.15.42:0/3496797865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 0x55f1769b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec27fc700 1 -- 172.21.15.42:0/3496797865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 msgr2=0x55f1769b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec27fc700 1 --2- 172.21.15.42:0/3496797865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 0x55f1769b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.968+0000 7fcec27fc700 1 -- 172.21.15.42:0/3496797865 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f176711c00 con 0x55f1769b04a0 2023-12-12T10:04:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.969+0000 7fcec27fc700 1 --2- 172.21.15.42:0/3496797865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b8780 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fceb4000c00 tx=0x7fceb40122e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e023994501bc54fe server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.969+0000 7fceb9ffb700 1 -- 172.21.15.42:0/3496797865 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4017070 con 0x55f1769b04a0 2023-12-12T10:04:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.969+0000 7fceb9ffb700 1 -- 172.21.15.42:0/3496797865 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fceb4018040 con 0x55f1769b04a0 2023-12-12T10:04:53.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.969+0000 7fceb9ffb700 1 -- 172.21.15.42:0/3496797865 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4003680 con 0x55f1769b04a0 2023-12-12T10:04:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.969+0000 7fced548db80 1 -- 172.21.15.42:0/3496797865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 msgr2=0x55f1769b8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.969+0000 7fced548db80 1 --2- 172.21.15.42:0/3496797865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b8780 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fceb4000c00 tx=0x7fceb40122e0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 -- 172.21.15.42:0/3496797865 shutdown_connections 2023-12-12T10:04:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 --2- 172.21.15.42:0/3496797865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769afb80 0x55f1769b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 --2- 172.21.15.42:0/3496797865 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769b42d0 0x55f1769b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 --2- 172.21.15.42:0/3496797865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b04a0 0x55f1769b8780 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 -- 172.21.15.42:0/3496797865 >> 172.21.15.42:0/3496797865 conn(0x55f1768da780 msgr2=0x55f1769a5bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:53.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 -- 172.21.15.42:0/3496797865 shutdown_connections 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 -- 172.21.15.42:0/3496797865 wait complete. 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 -- start start 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 Processor -- start 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 0x55f1769c0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 0x55f1769c0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769be520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1769beb40 con 0x55f1769b42d0 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1769becb0 con 0x55f1769b04a0 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1769bee70 con 0x55f1769afb80 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fced548db80 10 client.0 _ll_register_callbacks cb 0x55f1767e4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fcec37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769be520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fcec37fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769be520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36976/0 (socket says 172.21.15.42:36976) 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fcec37fe700 1 -- 172.21.15.42:0/4081501780 learned_addr learned my addr 172.21.15.42:0/4081501780 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:04:53.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.970+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 0x55f1769c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fcec27fc700 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 0x55f1769c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fcec37fe700 1 -- 172.21.15.42:0/4081501780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 msgr2=0x55f1769c0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fcec37fe700 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 0x55f1769c0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fcec37fe700 1 -- 172.21.15.42:0/4081501780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 msgr2=0x55f1769c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fcec37fe700 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 0x55f1769c0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fcec37fe700 1 -- 172.21.15.42:0/4081501780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f176728d80 con 0x55f1769b42d0 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fcec27fc700 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 0x55f1769c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fcec37fe700 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769be520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0ca0f0 tx=0x7fcebc099a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb2bf24ad7eb401 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fceaa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fceaa7fc700 1 -- 172.21.15.42:0/4081501780 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc005aa0 con 0x55f1769b42d0 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fceaa7fc700 1 -- 172.21.15.42:0/4081501780 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcebc099b30 con 0x55f1769b42d0 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fced548db80 1 -- 172.21.15.42:0/4081501780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcebc0a1c60 con 0x55f1769b42d0 2023-12-12T10:04:53.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fceaa7fc700 1 -- 172.21.15.42:0/4081501780 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcebc099cf0 con 0x55f1769b42d0 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fced548db80 10 client.16050 fetch_fsmap learned FSMap version 37 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fced548db80 1 -- 172.21.15.42:0/4081501780 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55f17674f1f0 con 0x55f1769b42d0 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.971+0000 7fceaa7fc700 1 -- 172.21.15.42:0/4081501780 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcebc063020 con 0x55f1769b42d0 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fceaa7fc700 1 -- 172.21.15.42:0/4081501780 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 37) v1 ==== 7970+0+0 (secure 0 0 0) 0x7fcebc0a17d0 con 0x55f1769b42d0 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fced548db80 10 client.16050 fetch_fsmap finished waiting for FSMap version 37 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fced548db80 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fced548db80 20 client.16050 populate_metadata read hostname 'smithi042' 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fced548db80 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f176a20c00 0x55f176a23020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fced548db80 4 client.16050 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fced548db80 1 -- 172.21.15.42:0/4081501780 --> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f1768c5f70 con 0x55f176a20c00 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f176a20c00 0x55f176a23020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fceaa7fc700 10 client.16050 ms_handle_connect on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:53.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:53.972+0000 7fcec2ffd700 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f176a20c00 0x55f176a23020 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1df226c211d51c18 server_cookie=3ca0dd7457d362aa in_seq=0 out_seq=0 2023-12-12T10:04:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.100+0000 7fceaa7fc700 1 -- 172.21.15.42:0/4081501780 <== mds.0 v2:172.21.15.111:6838/2430709127 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f1768c5f70 con 0x55f176a20c00 2023-12-12T10:04:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.100+0000 7fceaa7fc700 10 client.16050 handle_command_reply: tid=0 2023-12-12T10:04:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.100+0000 7fced548db80 1 client.16050 shutdown 2023-12-12T10:04:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.100+0000 7fced548db80 1 -- 172.21.15.42:0/4081501780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 msgr2=0x55f1769be520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.100+0000 7fced548db80 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769be520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0ca0f0 tx=0x7fcebc099a50 comp rx=0 tx=0).stop 2023-12-12T10:04:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fced548db80 1 -- 172.21.15.42:0/4081501780 shutdown_connections 2023-12-12T10:04:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fced548db80 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1769b42d0 0x55f1769be520 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:54.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fced548db80 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55f1769afb80 0x55f1769c0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fced548db80 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.111:6838/2430709127,v1:172.21.15.111:6839/2430709127] conn(0x55f176a20c00 0x55f176a23020 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fced548db80 1 --2- 172.21.15.42:0/4081501780 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f1769b04a0 0x55f1769c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fceaa7fc700 0 client.16050 ms_handle_reset on v2:172.21.15.111:6838/2430709127 2023-12-12T10:04:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fced548db80 1 -- 172.21.15.42:0/4081501780 >> 172.21.15.42:0/4081501780 conn(0x55f1768da780 msgr2=0x55f1769b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fced548db80 1 -- 172.21.15.42:0/4081501780 shutdown_connections 2023-12-12T10:04:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fced548db80 1 -- 172.21.15.42:0/4081501780 wait complete. 2023-12-12T10:04:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fced548db80 20 client.16050 trim_cache size 0 max 16384 2023-12-12T10:04:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2023-12-12T10:04:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2023-12-12T10:04:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2023-12-12T10:04:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2023-12-12T10:04:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 -- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fceb00a15f0 msgr2=0x7fceb00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fceb00a15f0 0x7fceb00a3ab0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fceac006010 tx=0x7fceac00b260 comp rx=0 tx=0).stop 2023-12-12T10:04:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 -- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 msgr2=0x7fcec4199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:04:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 0x7fcec4199230 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcebc007ee0 tx=0x7fcebc00d3b0 comp rx=0 tx=0).stop 2023-12-12T10:04:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 -- 172.21.15.42:0/3412108672 shutdown_connections 2023-12-12T10:04:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fceb00a15f0 0x7fceb00a3ab0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcec41003d0 0x7fcec4198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcec4106470 0x7fcec4199230 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 --2- 172.21.15.42:0/3412108672 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcec4100d90 0x7fcec4198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:04:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.101+0000 7fcec9294700 1 -- 172.21.15.42:0/3412108672 >> 172.21.15.42:0/3412108672 conn(0x7fcec40fbea0 msgr2=0x7fcec40fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:04:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.102+0000 7fcec9294700 1 -- 172.21.15.42:0/3412108672 shutdown_connections 2023-12-12T10:04:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2023-12-12T10:04:54.102+0000 7fcec9294700 1 -- 172.21.15.42:0/3412108672 wait complete. 2023-12-12T10:04:54.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:54 smithi042 ceph-mon[98894]: pgmap v191: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 79 KiB/s wr, 62 op/s 2023-12-12T10:04:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:54 smithi111 ceph-mon[102565]: pgmap v191: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 79 KiB/s wr, 62 op/s 2023-12-12T10:04:55.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:54 smithi173 ceph-mon[103308]: pgmap v191: 129 pgs: 129 active+clean; 716 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 79 KiB/s wr, 62 op/s 2023-12-12T10:04:56.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:56 smithi042 ceph-mon[98894]: pgmap v192: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 113 KiB/s wr, 58 op/s 2023-12-12T10:04:56.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:56 smithi111 ceph-mon[102565]: pgmap v192: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 113 KiB/s wr, 58 op/s 2023-12-12T10:04:57.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:56 smithi173 ceph-mon[103308]: pgmap v192: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 299 KiB/s rd, 113 KiB/s wr, 58 op/s 2023-12-12T10:04:58.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:58 smithi042 ceph-mon[98894]: pgmap v193: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 13 op/s 2023-12-12T10:04:58.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:58 smithi111 ceph-mon[102565]: pgmap v193: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 13 op/s 2023-12-12T10:04:59.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:58 smithi173 ceph-mon[103308]: pgmap v193: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 52 KiB/s wr, 13 op/s 2023-12-12T10:04:59.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:04:59 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:04:59.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:04:59 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:05:00.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:04:59 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:05:00.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:00 smithi042 ceph-mon[98894]: pgmap v194: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 109 op/s 2023-12-12T10:05:00.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:00 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:05:00.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:00 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:05:00.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:00 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:05:00.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:00 smithi111 ceph-mon[102565]: pgmap v194: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 109 op/s 2023-12-12T10:05:00.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:00 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:05:00.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:00 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:05:00.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:00 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:05:01.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:00 smithi173 ceph-mon[103308]: pgmap v194: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 109 op/s 2023-12-12T10:05:01.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:00 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:05:01.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:00 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:05:01.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:00 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:05:01.747 INFO:tasks.workunit.client.0.smithi042.stdout:ok 2023-12-12T10:05:01.751 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-12T10:05:01.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-12T10:05:01.801 INFO:tasks.workunit:Running workunit fs/misc/general_vxattrs.sh... 2023-12-12T10:05:01.802 DEBUG:teuthology.orchestra.run.smithi042:workunit test fs/misc/general_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh 2023-12-12T10:05:01.845 INFO:tasks.workunit.client.0.smithi042.stderr:+ mkdir -p dir 2023-12-12T10:05:01.847 INFO:tasks.workunit.client.0.smithi042.stderr:+ getfattr -n ceph.dir.pin dir 2023-12-12T10:05:01.847 INFO:tasks.workunit.client.0.smithi042.stderr:+ grep 'ceph.dir.pin="-1"' 2023-12-12T10:05:01.849 INFO:tasks.workunit.client.0.smithi042.stdout:ceph.dir.pin="-1" 2023-12-12T10:05:01.849 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n ceph.dir.pin dir 2023-12-12T10:05:01.849 INFO:tasks.workunit.client.0.smithi042.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-12T10:05:01.850 INFO:tasks.workunit.client.0.smithi042.stdout:setfattr: dir: Invalid argument 2023-12-12T10:05:01.850 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n ceph.dir.pin -v 1 dir 2023-12-12T10:05:01.852 INFO:tasks.workunit.client.0.smithi042.stderr:+ getfattr -n ceph.dir.pin dir 2023-12-12T10:05:01.852 INFO:tasks.workunit.client.0.smithi042.stderr:+ grep 'ceph.dir.pin="1"' 2023-12-12T10:05:01.854 INFO:tasks.workunit.client.0.smithi042.stdout:ceph.dir.pin="1" 2023-12-12T10:05:01.854 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -x ceph.dir.pin dir 2023-12-12T10:05:01.856 INFO:tasks.workunit.client.0.smithi042.stderr:+ getfattr -n ceph.dir.pin dir 2023-12-12T10:05:01.856 INFO:tasks.workunit.client.0.smithi042.stderr:+ grep 'ceph.dir.pin="-1"' 2023-12-12T10:05:01.857 INFO:tasks.workunit.client.0.smithi042.stdout:ceph.dir.pin="-1" 2023-12-12T10:05:01.857 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n ceph.dir.pin.distributed dir 2023-12-12T10:05:01.857 INFO:tasks.workunit.client.0.smithi042.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-12T10:05:01.858 INFO:tasks.workunit.client.0.smithi042.stdout:setfattr: dir: Invalid argument 2023-12-12T10:05:01.858 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 dir 2023-12-12T10:05:01.860 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -x ceph.dir.pin.distributed dir 2023-12-12T10:05:01.862 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n ceph.dir.pin.random dir 2023-12-12T10:05:01.862 INFO:tasks.workunit.client.0.smithi042.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-12T10:05:01.863 INFO:tasks.workunit.client.0.smithi042.stdout:setfattr: dir: Invalid argument 2023-12-12T10:05:01.863 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n ceph.dir.pin.random -v 0.01 dir 2023-12-12T10:05:01.864 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -x ceph.dir.pin.random dir 2023-12-12T10:05:01.866 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n ceph.dir.subvolume dir 2023-12-12T10:05:01.866 INFO:tasks.workunit.client.0.smithi042.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-12T10:05:01.867 INFO:tasks.workunit.client.0.smithi042.stdout:setfattr: dir: Invalid argument 2023-12-12T10:05:01.868 INFO:tasks.workunit.client.0.smithi042.stderr:+ setfattr -n ceph.dir.subvolume -v 1 dir 2023-12-12T10:05:01.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T10:05:01.877 INFO:tasks.workunit.client.0.smithi042.stderr:setfattr: dir: Invalid argument 2023-12-12T10:05:01.877 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2023-12-12T10:05:01.877 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-12-12T10:05:02.852 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi042 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-12-12T10:05:02.988 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=b02660a13c2240ecbf2796147cb87b96 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_f1fbf3c583d03e19f5e58eb618e78d8e10d075f9/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi042 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh' 2023-12-12T10:05:02.992 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-12-12T10:05:03.003 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T10:05:03.018 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:02 smithi111 ceph-mon[102565]: pgmap v195: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 101 op/s 2023-12-12T10:05:03.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:02 smithi042 ceph-mon[98894]: pgmap v195: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 101 op/s 2023-12-12T10:05:03.019 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-12-12T10:05:03.029 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-12-12T10:05:03.029 INFO:teuthology.task.exec:Executing custom commands... 2023-12-12T10:05:03.029 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T10:05:03.029 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-12-12T10:05:03.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:02 smithi173 ceph-mon[103308]: pgmap v195: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 101 op/s 2023-12-12T10:05:03.088 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-12-12T10:05:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:04 smithi042 ceph-mon[98894]: pgmap v196: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 102 op/s 2023-12-12T10:05:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:04 smithi111 ceph-mon[102565]: pgmap v196: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 102 op/s 2023-12-12T10:05:05.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:04 smithi173 ceph-mon[103308]: pgmap v196: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 102 op/s 2023-12-12T10:05:06.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:06 smithi042 ceph-mon[98894]: pgmap v197: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 74 KiB/s wr, 120 op/s 2023-12-12T10:05:06.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:06 smithi111 ceph-mon[102565]: pgmap v197: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 74 KiB/s wr, 120 op/s 2023-12-12T10:05:07.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:06 smithi173 ceph-mon[103308]: pgmap v197: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 74 KiB/s wr, 120 op/s 2023-12-12T10:05:08.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:08 smithi042 ceph-mon[98894]: pgmap v198: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 40 KiB/s wr, 114 op/s 2023-12-12T10:05:08.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:08 smithi111 ceph-mon[102565]: pgmap v198: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 40 KiB/s wr, 114 op/s 2023-12-12T10:05:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:08 smithi173 ceph-mon[103308]: pgmap v198: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 40 KiB/s wr, 114 op/s 2023-12-12T10:05:10.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:10 smithi042 ceph-mon[98894]: pgmap v199: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 119 op/s 2023-12-12T10:05:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:10 smithi111 ceph-mon[102565]: pgmap v199: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 119 op/s 2023-12-12T10:05:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:10 smithi173 ceph-mon[103308]: pgmap v199: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 43 KiB/s wr, 119 op/s 2023-12-12T10:05:12.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:12 smithi042 ceph-mon[98894]: pgmap v200: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 35 KiB/s wr, 23 op/s 2023-12-12T10:05:12.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:12 smithi111 ceph-mon[102565]: pgmap v200: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 35 KiB/s wr, 23 op/s 2023-12-12T10:05:13.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:12 smithi173 ceph-mon[103308]: pgmap v200: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 35 KiB/s wr, 23 op/s 2023-12-12T10:05:14.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:14 smithi042 ceph-mon[98894]: pgmap v201: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 35 KiB/s wr, 24 op/s 2023-12-12T10:05:14.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:14 smithi111 ceph-mon[102565]: pgmap v201: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 35 KiB/s wr, 24 op/s 2023-12-12T10:05:15.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:14 smithi173 ceph-mon[103308]: pgmap v201: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 35 KiB/s wr, 24 op/s 2023-12-12T10:05:16.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:16 smithi042 ceph-mon[98894]: pgmap v202: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 KiB/s wr, 29 op/s 2023-12-12T10:05:16.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:16 smithi111 ceph-mon[102565]: pgmap v202: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 KiB/s wr, 29 op/s 2023-12-12T10:05:17.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:16 smithi173 ceph-mon[103308]: pgmap v202: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 KiB/s wr, 29 op/s 2023-12-12T10:05:18.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:18 smithi042 ceph-mon[98894]: pgmap v203: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2023-12-12T10:05:18.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:18 smithi111 ceph-mon[102565]: pgmap v203: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2023-12-12T10:05:19.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:18 smithi173 ceph-mon[103308]: pgmap v203: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2023-12-12T10:05:20.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:20 smithi042 ceph-mon[98894]: pgmap v204: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2023-12-12T10:05:20.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:20 smithi111 ceph-mon[102565]: pgmap v204: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2023-12-12T10:05:21.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:20 smithi173 ceph-mon[103308]: pgmap v204: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 15 op/s 2023-12-12T10:05:22.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:22 smithi042 ceph-mon[98894]: pgmap v205: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-12T10:05:22.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:22 smithi111 ceph-mon[102565]: pgmap v205: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-12T10:05:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:22 smithi173 ceph-mon[103308]: pgmap v205: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-12T10:05:24.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:24 smithi042 ceph-mon[98894]: pgmap v206: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-12T10:05:24.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:24 smithi111 ceph-mon[102565]: pgmap v206: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-12T10:05:25.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:24 smithi173 ceph-mon[103308]: pgmap v206: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-12T10:05:26.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:26 smithi042 ceph-mon[98894]: pgmap v207: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2023-12-12T10:05:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:26 smithi111 ceph-mon[102565]: pgmap v207: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2023-12-12T10:05:27.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:26 smithi173 ceph-mon[103308]: pgmap v207: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2023-12-12T10:05:28.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:28 smithi042 ceph-mon[98894]: pgmap v208: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:28.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:28 smithi111 ceph-mon[102565]: pgmap v208: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:28 smithi173 ceph-mon[103308]: pgmap v208: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:30.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:30 smithi042 ceph-mon[98894]: pgmap v209: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:30.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:30 smithi111 ceph-mon[102565]: pgmap v209: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:31.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:30 smithi173 ceph-mon[103308]: pgmap v209: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:32.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:32 smithi042 ceph-mon[98894]: pgmap v210: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:32.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:32 smithi111 ceph-mon[102565]: pgmap v210: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:32 smithi173 ceph-mon[103308]: pgmap v210: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:34.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:34 smithi042 ceph-mon[98894]: pgmap v211: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:34.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:34 smithi111 ceph-mon[102565]: pgmap v211: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:35.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:34 smithi173 ceph-mon[103308]: pgmap v211: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:36.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:36 smithi111 ceph-mon[102565]: pgmap v212: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:37.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:36 smithi173 ceph-mon[103308]: pgmap v212: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:37.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:36 smithi042 ceph-mon[98894]: pgmap v212: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:38.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:38 smithi111 ceph-mon[102565]: pgmap v213: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:39.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:38 smithi173 ceph-mon[103308]: pgmap v213: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:38 smithi042 ceph-mon[98894]: pgmap v213: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:40.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:40 smithi111 ceph-mon[102565]: pgmap v214: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:41.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:40 smithi173 ceph-mon[103308]: pgmap v214: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:41.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:40 smithi042 ceph-mon[98894]: pgmap v214: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:42.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:42 smithi111 ceph-mon[102565]: pgmap v215: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:43.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:42 smithi173 ceph-mon[103308]: pgmap v215: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:43.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:42 smithi042 ceph-mon[98894]: pgmap v215: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:44.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:44 smithi111 ceph-mon[102565]: pgmap v216: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:45.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:44 smithi173 ceph-mon[103308]: pgmap v216: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:45.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:44 smithi042 ceph-mon[98894]: pgmap v216: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:46.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:46 smithi111 ceph-mon[102565]: pgmap v217: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:47.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:46 smithi173 ceph-mon[103308]: pgmap v217: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:47.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:46 smithi042 ceph-mon[98894]: pgmap v217: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:48.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:48 smithi111 ceph-mon[102565]: pgmap v218: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:49.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:48 smithi173 ceph-mon[103308]: pgmap v218: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:49.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:48 smithi042 ceph-mon[98894]: pgmap v218: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:50 smithi111 ceph-mon[102565]: pgmap v219: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:51.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:50 smithi173 ceph-mon[103308]: pgmap v219: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:51.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:50 smithi042 ceph-mon[98894]: pgmap v219: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:52 smithi111 ceph-mon[102565]: pgmap v220: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:52 smithi173 ceph-mon[103308]: pgmap v220: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:53.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:52 smithi042 ceph-mon[98894]: pgmap v220: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:54.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:54 smithi111 ceph-mon[102565]: pgmap v221: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:55.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:54 smithi173 ceph-mon[103308]: pgmap v221: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:55.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:54 smithi042 ceph-mon[98894]: pgmap v221: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:56.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:56 smithi111 ceph-mon[102565]: pgmap v222: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:57.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:56 smithi173 ceph-mon[103308]: pgmap v222: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:57.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:56 smithi042 ceph-mon[98894]: pgmap v222: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:05:58.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:05:58 smithi111 ceph-mon[102565]: pgmap v223: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:59.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:05:58 smithi173 ceph-mon[103308]: pgmap v223: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:05:59.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:05:58 smithi042 ceph-mon[98894]: pgmap v223: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:00.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:00 smithi111 ceph-mon[102565]: pgmap v224: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:00.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:00 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:06:01.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:00 smithi173 ceph-mon[103308]: pgmap v224: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:01.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:00 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:06:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:00 smithi042 ceph-mon[98894]: pgmap v224: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:01.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:00 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:06:01.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:01 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:06:01.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:01 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:06:01.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:01 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:06:02.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:01 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:06:02.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:01 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:06:02.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:01 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:06:02.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:01 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:06:02.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:01 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:06:02.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:01 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:06:02.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:02 smithi111 ceph-mon[102565]: pgmap v225: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:03.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:02 smithi173 ceph-mon[103308]: pgmap v225: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:03.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:02 smithi042 ceph-mon[98894]: pgmap v225: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:04.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:04 smithi111 ceph-mon[102565]: pgmap v226: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:05.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:04 smithi173 ceph-mon[103308]: pgmap v226: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:05.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:04 smithi042 ceph-mon[98894]: pgmap v226: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:06.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:06 smithi111 ceph-mon[102565]: pgmap v227: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-12-12T10:06:07.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:06 smithi173 ceph-mon[103308]: pgmap v227: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-12-12T10:06:07.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:06 smithi042 ceph-mon[98894]: pgmap v227: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-12-12T10:06:08.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:08 smithi111 ceph-mon[102565]: pgmap v228: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:06:08.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:08 smithi111 ceph-mon[102565]: osdmap e93: 12 total, 12 up, 12 in 2023-12-12T10:06:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:08 smithi173 ceph-mon[103308]: pgmap v228: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:06:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:08 smithi173 ceph-mon[103308]: osdmap e93: 12 total, 12 up, 12 in 2023-12-12T10:06:09.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:08 smithi042 ceph-mon[98894]: pgmap v228: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:06:09.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:08 smithi042 ceph-mon[98894]: osdmap e93: 12 total, 12 up, 12 in 2023-12-12T10:06:10.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:10 smithi111 ceph-mon[102565]: pgmap v230: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-12-12T10:06:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:10 smithi173 ceph-mon[103308]: pgmap v230: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-12-12T10:06:11.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:10 smithi042 ceph-mon[98894]: pgmap v230: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-12-12T10:06:12.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:12 smithi111 ceph-mon[102565]: pgmap v231: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-12-12T10:06:13.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:12 smithi173 ceph-mon[103308]: pgmap v231: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-12-12T10:06:13.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:12 smithi042 ceph-mon[98894]: pgmap v231: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-12-12T10:06:14.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:14 smithi111 ceph-mon[102565]: pgmap v232: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-12-12T10:06:15.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:14 smithi173 ceph-mon[103308]: pgmap v232: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-12-12T10:06:15.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:14 smithi042 ceph-mon[98894]: pgmap v232: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 29 op/s 2023-12-12T10:06:16.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:16 smithi111 ceph-mon[102565]: pgmap v233: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-12-12T10:06:17.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:16 smithi173 ceph-mon[103308]: pgmap v233: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-12-12T10:06:17.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:16 smithi042 ceph-mon[98894]: pgmap v233: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-12-12T10:06:18.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:18 smithi111 ceph-mon[102565]: pgmap v234: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-12-12T10:06:18.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:18 smithi111 ceph-mon[102565]: osdmap e94: 12 total, 12 up, 12 in 2023-12-12T10:06:19.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:18 smithi173 ceph-mon[103308]: pgmap v234: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-12-12T10:06:19.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:18 smithi173 ceph-mon[103308]: osdmap e94: 12 total, 12 up, 12 in 2023-12-12T10:06:19.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:18 smithi042 ceph-mon[98894]: pgmap v234: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 307 B/s wr, 11 op/s 2023-12-12T10:06:19.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:18 smithi042 ceph-mon[98894]: osdmap e94: 12 total, 12 up, 12 in 2023-12-12T10:06:20.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:20 smithi111 ceph-mon[102565]: pgmap v236: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:06:21.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:20 smithi173 ceph-mon[103308]: pgmap v236: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:06:21.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:20 smithi042 ceph-mon[98894]: pgmap v236: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:06:22.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:22 smithi111 ceph-mon[102565]: pgmap v237: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:06:23.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:22 smithi173 ceph-mon[103308]: pgmap v237: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:06:23.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:22 smithi042 ceph-mon[98894]: pgmap v237: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:06:24.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:24 smithi111 ceph-mon[102565]: pgmap v238: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:06:25.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:24 smithi173 ceph-mon[103308]: pgmap v238: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:06:25.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:24 smithi042 ceph-mon[98894]: pgmap v238: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:06:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:26 smithi111 ceph-mon[102565]: pgmap v239: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:06:27.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:26 smithi173 ceph-mon[103308]: pgmap v239: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:06:27.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:26 smithi042 ceph-mon[98894]: pgmap v239: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:06:28.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:28 smithi111 ceph-mon[102565]: pgmap v240: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:06:29.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:28 smithi173 ceph-mon[103308]: pgmap v240: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:06:29.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:28 smithi042 ceph-mon[98894]: pgmap v240: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:06:30.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:30 smithi111 ceph-mon[102565]: pgmap v241: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 op/s 2023-12-12T10:06:31.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:30 smithi173 ceph-mon[103308]: pgmap v241: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 op/s 2023-12-12T10:06:31.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:30 smithi042 ceph-mon[98894]: pgmap v241: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 op/s 2023-12-12T10:06:33.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:32 smithi173 ceph-mon[103308]: pgmap v242: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-12-12T10:06:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:32 smithi042 ceph-mon[98894]: pgmap v242: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-12-12T10:06:33.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:32 smithi111 ceph-mon[102565]: pgmap v242: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-12-12T10:06:35.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:34 smithi173 ceph-mon[103308]: pgmap v243: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-12-12T10:06:35.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:34 smithi042 ceph-mon[98894]: pgmap v243: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-12-12T10:06:35.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:34 smithi111 ceph-mon[102565]: pgmap v243: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-12-12T10:06:37.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:36 smithi173 ceph-mon[103308]: pgmap v244: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 13 op/s 2023-12-12T10:06:37.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:36 smithi042 ceph-mon[98894]: pgmap v244: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 13 op/s 2023-12-12T10:06:37.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:36 smithi111 ceph-mon[102565]: pgmap v244: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 13 op/s 2023-12-12T10:06:39.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:38 smithi173 ceph-mon[103308]: pgmap v245: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-12-12T10:06:39.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:38 smithi042 ceph-mon[98894]: pgmap v245: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-12-12T10:06:39.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:38 smithi111 ceph-mon[102565]: pgmap v245: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2023-12-12T10:06:41.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:40 smithi173 ceph-mon[103308]: pgmap v246: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 13 op/s 2023-12-12T10:06:41.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:40 smithi042 ceph-mon[98894]: pgmap v246: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 13 op/s 2023-12-12T10:06:41.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:40 smithi111 ceph-mon[102565]: pgmap v246: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 13 op/s 2023-12-12T10:06:43.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:42 smithi173 ceph-mon[103308]: pgmap v247: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:43.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:42 smithi042 ceph-mon[98894]: pgmap v247: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:43.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:42 smithi111 ceph-mon[102565]: pgmap v247: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:45.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:44 smithi173 ceph-mon[103308]: pgmap v248: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:45.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:44 smithi042 ceph-mon[98894]: pgmap v248: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:45.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:44 smithi111 ceph-mon[102565]: pgmap v248: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:47.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:46 smithi173 ceph-mon[103308]: pgmap v249: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:47.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:46 smithi042 ceph-mon[98894]: pgmap v249: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:47.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:46 smithi111 ceph-mon[102565]: pgmap v249: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:49.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:48 smithi173 ceph-mon[103308]: pgmap v250: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:49.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:48 smithi042 ceph-mon[98894]: pgmap v250: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:49.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:48 smithi111 ceph-mon[102565]: pgmap v250: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:51.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:50 smithi173 ceph-mon[103308]: pgmap v251: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:51.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:50 smithi042 ceph-mon[98894]: pgmap v251: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:51.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:50 smithi111 ceph-mon[102565]: pgmap v251: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:52 smithi173 ceph-mon[103308]: pgmap v252: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:53.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:52 smithi042 ceph-mon[98894]: pgmap v252: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:53.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:52 smithi111 ceph-mon[102565]: pgmap v252: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:55.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:54 smithi173 ceph-mon[103308]: pgmap v253: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:55.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:54 smithi042 ceph-mon[98894]: pgmap v253: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:55.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:54 smithi111 ceph-mon[102565]: pgmap v253: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:57.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:56 smithi173 ceph-mon[103308]: pgmap v254: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:57.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:56 smithi042 ceph-mon[98894]: pgmap v254: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:57.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:56 smithi111 ceph-mon[102565]: pgmap v254: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:06:59.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:06:58 smithi173 ceph-mon[103308]: pgmap v255: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:59.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:06:58 smithi042 ceph-mon[98894]: pgmap v255: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:06:59.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:06:58 smithi111 ceph-mon[102565]: pgmap v255: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:01.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:00 smithi173 ceph-mon[103308]: pgmap v256: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:01.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:00 smithi042 ceph-mon[98894]: pgmap v256: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:01.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:00 smithi111 ceph-mon[102565]: pgmap v256: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:02.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:01 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:07:02.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:01 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:07:02.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:01 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:07:03.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:02 smithi173 ceph-mon[103308]: pgmap v257: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:03.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:02 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:07:03.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:02 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:07:03.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:02 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:07:03.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:02 smithi042 ceph-mon[98894]: pgmap v257: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:03.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:02 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:07:03.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:02 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:07:03.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:02 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:07:03.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:02 smithi111 ceph-mon[102565]: pgmap v257: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:03.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:02 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:07:03.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:02 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:07:03.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:02 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:07:05.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:04 smithi173 ceph-mon[103308]: pgmap v258: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:05.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:04 smithi042 ceph-mon[98894]: pgmap v258: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:05.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:04 smithi111 ceph-mon[102565]: pgmap v258: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:07.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:06 smithi173 ceph-mon[103308]: pgmap v259: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-12-12T10:07:07.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:06 smithi042 ceph-mon[98894]: pgmap v259: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-12-12T10:07:07.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:06 smithi111 ceph-mon[102565]: pgmap v259: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-12-12T10:07:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:08 smithi173 ceph-mon[103308]: pgmap v260: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:07:09.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:08 smithi173 ceph-mon[103308]: osdmap e95: 12 total, 12 up, 12 in 2023-12-12T10:07:09.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:08 smithi042 ceph-mon[98894]: pgmap v260: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:07:09.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:08 smithi042 ceph-mon[98894]: osdmap e95: 12 total, 12 up, 12 in 2023-12-12T10:07:09.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:08 smithi111 ceph-mon[102565]: pgmap v260: 129 pgs: 129 active+clean; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:07:09.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:08 smithi111 ceph-mon[102565]: osdmap e95: 12 total, 12 up, 12 in 2023-12-12T10:07:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:10 smithi173 ceph-mon[103308]: pgmap v262: 129 pgs: 23 active+clean+snaptrim, 36 active+clean, 70 active+clean+snaptrim_wait; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:07:11.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:10 smithi042 ceph-mon[98894]: pgmap v262: 129 pgs: 23 active+clean+snaptrim, 36 active+clean, 70 active+clean+snaptrim_wait; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:07:11.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:10 smithi111 ceph-mon[102565]: pgmap v262: 129 pgs: 23 active+clean+snaptrim, 36 active+clean, 70 active+clean+snaptrim_wait; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:07:13.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:12 smithi173 ceph-mon[103308]: pgmap v263: 129 pgs: 23 active+clean+snaptrim, 36 active+clean, 70 active+clean+snaptrim_wait; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:07:13.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:12 smithi042 ceph-mon[98894]: pgmap v263: 129 pgs: 23 active+clean+snaptrim, 36 active+clean, 70 active+clean+snaptrim_wait; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:07:13.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:12 smithi111 ceph-mon[102565]: pgmap v263: 129 pgs: 23 active+clean+snaptrim, 36 active+clean, 70 active+clean+snaptrim_wait; 717 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:07:14.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:13 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:14.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:13 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:14.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:13 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:15.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:14 smithi042 ceph-mon[98894]: pgmap v264: 129 pgs: 19 active+clean+snaptrim, 50 active+clean, 60 active+clean+snaptrim_wait; 717 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:07:15.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:14 smithi111 ceph-mon[102565]: pgmap v264: 129 pgs: 19 active+clean+snaptrim, 50 active+clean, 60 active+clean+snaptrim_wait; 717 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:07:15.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:14 smithi173 ceph-mon[103308]: pgmap v264: 129 pgs: 19 active+clean+snaptrim, 50 active+clean, 60 active+clean+snaptrim_wait; 717 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:07:17.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:16 smithi042 ceph-mon[98894]: pgmap v265: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:07:17.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:16 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:17.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:16 smithi111 ceph-mon[102565]: pgmap v265: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:07:17.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:16 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:17.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:16 smithi173 ceph-mon[103308]: pgmap v265: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:07:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:16 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:19.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:18 smithi042 ceph-mon[98894]: pgmap v266: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:07:19.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:18 smithi042 ceph-mon[98894]: osdmap e96: 12 total, 12 up, 12 in 2023-12-12T10:07:19.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:18 smithi111 ceph-mon[102565]: pgmap v266: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:07:19.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:18 smithi111 ceph-mon[102565]: osdmap e96: 12 total, 12 up, 12 in 2023-12-12T10:07:19.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:18 smithi173 ceph-mon[103308]: pgmap v266: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:07:19.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:18 smithi173 ceph-mon[103308]: osdmap e96: 12 total, 12 up, 12 in 2023-12-12T10:07:21.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:20 smithi042 ceph-mon[98894]: pgmap v268: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:07:21.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:20 smithi111 ceph-mon[102565]: pgmap v268: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:07:21.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:20 smithi173 ceph-mon[103308]: pgmap v268: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:07:23.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:22 smithi042 ceph-mon[98894]: pgmap v269: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:07:23.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:22 smithi111 ceph-mon[102565]: pgmap v269: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:07:23.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:22 smithi173 ceph-mon[103308]: pgmap v269: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:07:24.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:23 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:24.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:23 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:24.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:23 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:25.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:24 smithi042 ceph-mon[98894]: pgmap v270: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:07:25.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:24 smithi111 ceph-mon[102565]: pgmap v270: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:07:25.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:24 smithi173 ceph-mon[103308]: pgmap v270: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 11 op/s 2023-12-12T10:07:27.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:26 smithi042 ceph-mon[98894]: pgmap v271: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:07:27.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:26 smithi111 ceph-mon[102565]: pgmap v271: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:07:27.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:26 smithi173 ceph-mon[103308]: pgmap v271: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:07:29.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:28 smithi042 ceph-mon[98894]: pgmap v272: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:07:29.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:28 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:29.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:28 smithi111 ceph-mon[102565]: pgmap v272: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:07:29.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:28 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:29.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:28 smithi173 ceph-mon[103308]: pgmap v272: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:07:29.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:28 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:30.153 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:29 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:30.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:29 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:30.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:29 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:31.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:30 smithi042 ceph-mon[98894]: pgmap v273: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 11 op/s 2023-12-12T10:07:31.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:30 smithi111 ceph-mon[102565]: pgmap v273: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 11 op/s 2023-12-12T10:07:31.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:30 smithi173 ceph-mon[103308]: pgmap v273: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 11 op/s 2023-12-12T10:07:33.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:32 smithi042 ceph-mon[98894]: pgmap v274: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:33.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:32 smithi111 ceph-mon[102565]: pgmap v274: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:33.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:32 smithi173 ceph-mon[103308]: pgmap v274: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:35.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:34 smithi042 ceph-mon[98894]: pgmap v275: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:35.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:34 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:35.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:34 smithi111 ceph-mon[102565]: pgmap v275: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:35.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:34 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:35.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:34 smithi173 ceph-mon[103308]: pgmap v275: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:35.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:34 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:37.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:36 smithi042 ceph-mon[98894]: pgmap v276: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:37.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:36 smithi111 ceph-mon[102565]: pgmap v276: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:37.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:36 smithi173 ceph-mon[103308]: pgmap v276: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:38.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:37 smithi111 ceph-mon[102565]: pgmap v277: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:38.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:37 smithi173 ceph-mon[103308]: pgmap v277: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:38.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:37 smithi042 ceph-mon[98894]: pgmap v277: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:40.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:40 smithi111 ceph-mon[102565]: pgmap v278: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:40.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:40 smithi173 ceph-mon[103308]: pgmap v278: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:40.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:40 smithi042 ceph-mon[98894]: pgmap v278: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:42.750 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:42 smithi111 ceph-mon[102565]: pgmap v279: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:42.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:42 smithi173 ceph-mon[103308]: pgmap v279: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:42.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:42 smithi042 ceph-mon[98894]: pgmap v279: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:44.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:44 smithi111 ceph-mon[102565]: pgmap v280: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:44.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:44 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:44 smithi173 ceph-mon[103308]: pgmap v280: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:44.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:44 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:44.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:44 smithi042 ceph-mon[98894]: pgmap v280: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:44.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:44 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:46.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:46 smithi111 ceph-mon[102565]: pgmap v281: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:46.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:46 smithi173 ceph-mon[103308]: pgmap v281: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:46.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:46 smithi042 ceph-mon[98894]: pgmap v281: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:48.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:48 smithi111 ceph-mon[102565]: pgmap v282: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:48.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:48 smithi173 ceph-mon[103308]: pgmap v282: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:48.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:48 smithi042 ceph-mon[98894]: pgmap v282: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:50.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:50 smithi111 ceph-mon[102565]: pgmap v283: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:50.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:50 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:50.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:50 smithi173 ceph-mon[103308]: pgmap v283: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:50.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:50 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:50.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:50 smithi042 ceph-mon[98894]: pgmap v283: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:50.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:50 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:51.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:51 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:51.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:51 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:51.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:51 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:52.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:52 smithi111 ceph-mon[102565]: pgmap v284: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:52.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:52 smithi173 ceph-mon[103308]: pgmap v284: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:52.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:52 smithi042 ceph-mon[98894]: pgmap v284: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:54.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:54 smithi111 ceph-mon[102565]: pgmap v285: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:54.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:54 smithi173 ceph-mon[103308]: pgmap v285: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:54.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:54 smithi042 ceph-mon[98894]: pgmap v285: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:56.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:56 smithi111 ceph-mon[102565]: pgmap v286: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:56.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:56 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:56 smithi173 ceph-mon[103308]: pgmap v286: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:56 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:56.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:56 smithi042 ceph-mon[98894]: pgmap v286: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:07:56.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:56 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:07:58.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:07:58 smithi111 ceph-mon[102565]: pgmap v287: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:58.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:07:58 smithi173 ceph-mon[103308]: pgmap v287: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:07:58.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:07:58 smithi042 ceph-mon[98894]: pgmap v287: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:00.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:00 smithi111 ceph-mon[102565]: pgmap v288: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:00.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:00 smithi173 ceph-mon[103308]: pgmap v288: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:00.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:00 smithi042 ceph-mon[98894]: pgmap v288: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:02.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:02 smithi173 ceph-mon[103308]: pgmap v289: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:02.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:02 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:08:02.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:02 smithi042 ceph-mon[98894]: pgmap v289: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:02.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:02 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:08:02.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:02 smithi111 ceph-mon[102565]: pgmap v289: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:02.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:02 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:08:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:03 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:08:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:03 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:08:03.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:03 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:08:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:03 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:08:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:03 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:08:03.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:03 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:08:03.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:03 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:08:03.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:03 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:08:03.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:03 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:08:04.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:04 smithi111 ceph-mon[102565]: pgmap v290: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:05.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:04 smithi173 ceph-mon[103308]: pgmap v290: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:05.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:04 smithi042 ceph-mon[98894]: pgmap v290: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:06.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:06 smithi111 ceph-mon[102565]: pgmap v291: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-12-12T10:08:07.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:06 smithi173 ceph-mon[103308]: pgmap v291: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-12-12T10:08:07.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:06 smithi042 ceph-mon[98894]: pgmap v291: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 28 op/s 2023-12-12T10:08:08.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:08 smithi111 ceph-mon[102565]: pgmap v292: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:08:08.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:08 smithi111 ceph-mon[102565]: osdmap e97: 12 total, 12 up, 12 in 2023-12-12T10:08:09.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:08 smithi173 ceph-mon[103308]: pgmap v292: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:08:09.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:08 smithi173 ceph-mon[103308]: osdmap e97: 12 total, 12 up, 12 in 2023-12-12T10:08:09.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:08 smithi042 ceph-mon[98894]: pgmap v292: 129 pgs: 129 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:08:09.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:08 smithi042 ceph-mon[98894]: osdmap e97: 12 total, 12 up, 12 in 2023-12-12T10:08:09.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:09 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:10.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:09 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:10.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:09 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:11.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:10 smithi173 ceph-mon[103308]: pgmap v294: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:08:11.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:10 smithi042 ceph-mon[98894]: pgmap v294: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:08:11.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:10 smithi111 ceph-mon[102565]: pgmap v294: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:08:13.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:12 smithi173 ceph-mon[103308]: pgmap v295: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:08:13.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:12 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:13.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:12 smithi042 ceph-mon[98894]: pgmap v295: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:08:13.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:12 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:13.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:12 smithi111 ceph-mon[102565]: pgmap v295: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:08:13.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:12 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:15.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:14 smithi173 ceph-mon[103308]: pgmap v296: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:08:15.157 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:14 smithi042 ceph-mon[98894]: pgmap v296: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:08:15.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:14 smithi111 ceph-mon[102565]: pgmap v296: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:08:17.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:16 smithi042 ceph-mon[98894]: pgmap v297: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:08:17.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:16 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:17.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:16 smithi111 ceph-mon[102565]: pgmap v297: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:08:17.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:16 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:17.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:16 smithi173 ceph-mon[103308]: pgmap v297: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:08:17.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:16 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:19.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:18 smithi042 ceph-mon[98894]: pgmap v298: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:08:19.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:18 smithi111 ceph-mon[102565]: pgmap v298: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:08:19.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:18 smithi173 ceph-mon[103308]: pgmap v298: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:08:21.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:20 smithi042 ceph-mon[98894]: pgmap v299: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 180 B/s wr, 14 op/s 2023-12-12T10:08:21.247 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:20 smithi111 ceph-mon[102565]: pgmap v299: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 180 B/s wr, 14 op/s 2023-12-12T10:08:21.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:20 smithi173 ceph-mon[103308]: pgmap v299: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 180 B/s wr, 14 op/s 2023-12-12T10:08:23.159 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:22 smithi042 ceph-mon[98894]: pgmap v300: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:08:23.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:22 smithi111 ceph-mon[102565]: pgmap v300: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:08:23.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:22 smithi173 ceph-mon[103308]: pgmap v300: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:08:25.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:24 smithi042 ceph-mon[98894]: pgmap v301: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:08:25.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:24 smithi111 ceph-mon[102565]: pgmap v301: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:08:25.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:24 smithi173 ceph-mon[103308]: pgmap v301: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:08:27.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:26 smithi111 ceph-mon[102565]: pgmap v302: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2023-12-12T10:08:27.249 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:26 smithi111 ceph-mon[102565]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:27.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:26 smithi173 ceph-mon[103308]: pgmap v302: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2023-12-12T10:08:27.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:26 smithi173 ceph-mon[103308]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:27.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:26 smithi042 ceph-mon[98894]: pgmap v302: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 13 op/s 2023-12-12T10:08:27.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:26 smithi042 ceph-mon[98894]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2023-12-12T10:08:28.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:27 smithi111 ceph-mon[102565]: pgmap v303: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:28.327 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:27 smithi173 ceph-mon[103308]: pgmap v303: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:28.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:27 smithi042 ceph-mon[98894]: pgmap v303: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:30.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:30 smithi173 ceph-mon[103308]: pgmap v304: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 717 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:30.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:30 smithi042 ceph-mon[98894]: pgmap v304: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 717 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:30.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:30 smithi111 ceph-mon[102565]: pgmap v304: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 717 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:32.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:32 smithi173 ceph-mon[103308]: pgmap v305: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 717 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:32.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:32 smithi042 ceph-mon[98894]: pgmap v305: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 717 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:32.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:32 smithi111 ceph-mon[102565]: pgmap v305: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 717 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:34.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:34 smithi173 ceph-mon[103308]: pgmap v306: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 717 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:34 smithi042 ceph-mon[98894]: pgmap v306: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 717 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:34.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:34 smithi111 ceph-mon[102565]: pgmap v306: 129 pgs: 20 active+clean+snaptrim, 62 active+clean+snaptrim_wait, 47 active+clean; 717 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:36.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:36 smithi173 ceph-mon[103308]: pgmap v307: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:36.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:36 smithi042 ceph-mon[98894]: pgmap v307: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:36.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:36 smithi111 ceph-mon[102565]: pgmap v307: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:38.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:38 smithi173 ceph-mon[103308]: pgmap v308: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:38.910 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:38 smithi042 ceph-mon[98894]: pgmap v308: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:38.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:38 smithi111 ceph-mon[102565]: pgmap v308: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:39.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-12T10:08:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.430+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3720550920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee4103f20 msgr2=0x7f2ee4104380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.430+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3720550920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee4103f20 0x7f2ee4104380 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2edc01c0d0 tx=0x7f2edc01c3e0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.448+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3720550920 shutdown_connections 2023-12-12T10:08:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.448+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3720550920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee4103f20 0x7f2ee4104380 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.448+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3720550920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ee4106420 0x7f2ee4069750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.451 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.448+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3720550920 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ee4069c90 0x7f2ee406a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.448+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3720550920 >> 172.21.15.42:0/3720550920 conn(0x7f2ee4077570 msgr2=0x7f2ee4077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:39.452 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.450+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3720550920 shutdown_connections 2023-12-12T10:08:39.457 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.455+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3720550920 wait complete. 2023-12-12T10:08:39.458 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.455+0000 7f2eeb38e700 1 Processor -- start 2023-12-12T10:08:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2eeb38e700 1 -- start start 2023-12-12T10:08:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2eeb38e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ee4069c90 0x7f2ee41106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2eeb38e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee4103f20 0x7f2ee4110c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2eeb38e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ee4106420 0x7f2ee419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2eeb38e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee419b800 con 0x7f2ee4103f20 2023-12-12T10:08:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2eeb38e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee419b970 con 0x7f2ee4106420 2023-12-12T10:08:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2eeb38e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee419bb30 con 0x7f2ee4069c90 2023-12-12T10:08:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2ee912a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ee4069c90 0x7f2ee41106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2ee912a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ee4069c90 0x7f2ee41106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:51970/0 (socket says 172.21.15.42:51970) 2023-12-12T10:08:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.461+0000 7f2ee912a700 1 -- 172.21.15.42:0/3349950242 learned_addr learned my addr 172.21.15.42:0/3349950242 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:08:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2ee992b700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ee4106420 0x7f2ee419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2ee912a700 1 -- 172.21.15.42:0/3349950242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ee4106420 msgr2=0x7f2ee419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2ee912a700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ee4106420 0x7f2ee419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2ee912a700 1 -- 172.21.15.42:0/3349950242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee4103f20 msgr2=0x7f2ee4110c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2ee912a700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee4103f20 0x7f2ee4110c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2ee912a700 1 -- 172.21.15.42:0/3349950242 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc01ad10 con 0x7f2ee4069c90 2023-12-12T10:08:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2ee992b700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ee4106420 0x7f2ee419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-12T10:08:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2ee912a700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ee4069c90 0x7f2ee41106f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e9a0 tx=0x7f2ee000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2eda7fc700 1 -- 172.21.15.42:0/3349950242 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0009790 con 0x7f2ee4069c90 2023-12-12T10:08:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2eda7fc700 1 -- 172.21.15.42:0/3349950242 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ee000cbd0 con 0x7f2ee4069c90 2023-12-12T10:08:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee419be10 con 0x7f2ee4069c90 2023-12-12T10:08:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.462+0000 7f2eda7fc700 1 -- 172.21.15.42:0/3349950242 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0018680 con 0x7f2ee4069c90 2023-12-12T10:08:39.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.463+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee406b1d0 con 0x7f2ee4069c90 2023-12-12T10:08:39.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.465+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee4100a00 con 0x7f2ee4069c90 2023-12-12T10:08:39.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.466+0000 7f2eda7fc700 1 -- 172.21.15.42:0/3349950242 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f2ee00187e0 con 0x7f2ee4069c90 2023-12-12T10:08:39.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.466+0000 7f2eda7fc700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f2ed00a1630 0x7f2ed00a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:39.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.467+0000 7f2eda7fc700 1 -- 172.21.15.42:0/3349950242 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2ee0014070 con 0x7f2ee4069c90 2023-12-12T10:08:39.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.467+0000 7f2ee8929700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f2ed00a1630 0x7f2ed00a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:39.478 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.468+0000 7f2ee8929700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f2ed00a1630 0x7f2ed00a3af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005950 tx=0x7f2ed40058e0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:39.486 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.483+0000 7f2eda7fc700 1 -- 172.21.15.42:0/3349950242 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f2ee00522f0 con 0x7f2ee4069c90 2023-12-12T10:08:39.895 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.892+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 --> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ee4111a90 con 0x7f2ed00a1630 2023-12-12T10:08:39.957 INFO:teuthology.orchestra.run.smithi042.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2023-12-12T00:00:00", "created": "2023-12-12T09:58:37", "first": "2023-12-12T09:59:00", "last": "2023-12-12T10:08:00", "last_pruned": "2023-12-12T10:08:00", "created_count": 10, "pruned_count": 3, "active": true} 2023-12-12T10:08:39.957 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.952+0000 7f2eda7fc700 1 -- 172.21.15.42:0/3349950242 <== mgr.15528 v2:172.21.15.42:6800/1776845983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f2ee4111a90 con 0x7f2ed00a1630 2023-12-12T10:08:39.960 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.957+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f2ed00a1630 msgr2=0x7f2ed00a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:39.960 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.957+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f2ed00a1630 0x7f2ed00a3af0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005950 tx=0x7f2ed40058e0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.960 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.957+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ee4069c90 msgr2=0x7f2ee41106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:39.961 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.957+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ee4069c90 0x7f2ee41106f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e9a0 tx=0x7f2ee000ed60 comp rx=0 tx=0).stop 2023-12-12T10:08:39.962 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.959+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 shutdown_connections 2023-12-12T10:08:39.963 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.959+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f2ed00a1630 0x7f2ed00a3af0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005950 tx=0x7f2ed40058e0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.963 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.959+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2ee4106420 0x7f2ee419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.963 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.959+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ee4069c90 0x7f2ee41106f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.963 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.959+0000 7f2eeb38e700 1 --2- 172.21.15.42:0/3349950242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2ee4103f20 0x7f2ee4110c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:39.963 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.959+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 >> 172.21.15.42:0/3349950242 conn(0x7f2ee4077570 msgr2=0x7f2ee4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:39.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.964+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 shutdown_connections 2023-12-12T10:08:39.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:39.968+0000 7f2eeb38e700 1 -- 172.21.15.42:0/3349950242 wait complete. 2023-12-12T10:08:39.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-12T10:08:40.249 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.247+0000 7fd73f9b3700 1 -- 172.21.15.42:0/723675468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 msgr2=0x7fd738107f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:40.250 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.247+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/723675468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 0x7fd738107f60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b0a0 tx=0x7fd73000b3b0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.253+0000 7fd73f9b3700 1 -- 172.21.15.42:0/723675468 shutdown_connections 2023-12-12T10:08:40.255 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.253+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/723675468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd7381084a0 0x7fd7381149b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.253+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/723675468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd73810db80 0x7fd73810df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.253+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/723675468 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 0x7fd738107f60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.256 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.253+0000 7fd73f9b3700 1 -- 172.21.15.42:0/723675468 >> 172.21.15.42:0/723675468 conn(0x7fd7380772d0 msgr2=0x7fd7380796f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:40.257 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.255+0000 7fd73f9b3700 1 -- 172.21.15.42:0/723675468 shutdown_connections 2023-12-12T10:08:40.259 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.257+0000 7fd73f9b3700 1 -- 172.21.15.42:0/723675468 wait complete. 2023-12-12T10:08:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.257+0000 7fd73f9b3700 1 Processor -- start 2023-12-12T10:08:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73f9b3700 1 -- start start 2023-12-12T10:08:40.261 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73f9b3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 0x7fd73819e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:40.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73f9b3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7381084a0 0x7fd73819e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:40.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73f9b3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd73810db80 0x7fd7381a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:40.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73f9b3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381a2f90 con 0x7fd73810db80 2023-12-12T10:08:40.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73f9b3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381a3100 con 0x7fd738107ae0 2023-12-12T10:08:40.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73f9b3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381a32c0 con 0x7fd7381084a0 2023-12-12T10:08:40.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73d74f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 0x7fd73819e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:40.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73d74f700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 0x7fd73819e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:40058/0 (socket says 172.21.15.42:40058) 2023-12-12T10:08:40.262 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.258+0000 7fd73d74f700 1 -- 172.21.15.42:0/3450422622 learned_addr learned my addr 172.21.15.42:0/3450422622 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:08:40.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.259+0000 7fd73d74f700 1 -- 172.21.15.42:0/3450422622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7381084a0 msgr2=0x7fd73819e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:40.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.259+0000 7fd73d74f700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7381084a0 0x7fd73819e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.259+0000 7fd73d74f700 1 -- 172.21.15.42:0/3450422622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd73810db80 msgr2=0x7fd7381a2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:40.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.259+0000 7fd73d74f700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd73810db80 0x7fd7381a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.259+0000 7fd73d74f700 1 -- 172.21.15.42:0/3450422622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd730009d00 con 0x7fd738107ae0 2023-12-12T10:08:40.265 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.260+0000 7fd73d74f700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 0x7fd73819e070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd72800b550 tx=0x7fd72800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:40.267 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.263+0000 7fd72e7fc700 1 -- 172.21.15.42:0/3450422622 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728011830 con 0x7fd738107ae0 2023-12-12T10:08:40.267 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.264+0000 7fd72e7fc700 1 -- 172.21.15.42:0/3450422622 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd728011e50 con 0x7fd738107ae0 2023-12-12T10:08:40.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.264+0000 7fd72e7fc700 1 -- 172.21.15.42:0/3450422622 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72800f2e0 con 0x7fd738107ae0 2023-12-12T10:08:40.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.264+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7381a35a0 con 0x7fd738107ae0 2023-12-12T10:08:40.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.264+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73807c010 con 0x7fd738107ae0 2023-12-12T10:08:40.268 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.266+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7381a37c0 con 0x7fd738107ae0 2023-12-12T10:08:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.269+0000 7fd72e7fc700 1 -- 172.21.15.42:0/3450422622 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fd728010610 con 0x7fd738107ae0 2023-12-12T10:08:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.270+0000 7fd72e7fc700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd7240a1800 0x7fd7240a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:40.272 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.270+0000 7fd72e7fc700 1 -- 172.21.15.42:0/3450422622 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd7280ccf60 con 0x7fd738107ae0 2023-12-12T10:08:40.276 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.274+0000 7fd73cf4e700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd7240a1800 0x7fd7240a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:40.281 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.274+0000 7fd72e7fc700 1 -- 172.21.15.42:0/3450422622 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fd7280fa2f0 con 0x7fd738107ae0 2023-12-12T10:08:40.282 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.279+0000 7fd73cf4e700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd7240a1800 0x7fd7240a3cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b070 tx=0x7fd730006000 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:40.696 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.693+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 --> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd7381a3790 con 0x7fd7240a1800 2023-12-12T10:08:40.738 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.735+0000 7fd72e7fc700 1 -- 172.21.15.42:0/3450422622 <== mgr.15528 v2:172.21.15.42:6800/1776845983 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7fd7381a3790 con 0x7fd7240a1800 2023-12-12T10:08:40.743 INFO:teuthology.orchestra.run.smithi042.stdout:/ 1m 6m3h 2023-12-12T10:08:40.747 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd7240a1800 msgr2=0x7fd7240a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:40.747 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd7240a1800 0x7fd7240a3cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd73000b070 tx=0x7fd730006000 comp rx=0 tx=0).stop 2023-12-12T10:08:40.748 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 msgr2=0x7fd73819e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:40.748 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 0x7fd73819e070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd72800b550 tx=0x7fd72800b910 comp rx=0 tx=0).stop 2023-12-12T10:08:40.748 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 shutdown_connections 2023-12-12T10:08:40.748 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fd7240a1800 0x7fd7240a3cc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.748 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd738107ae0 0x7fd73819e070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.748 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd73810db80 0x7fd7381a2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.748 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 --2- 172.21.15.42:0/3450422622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7381084a0 0x7fd73819e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:40.748 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 >> 172.21.15.42:0/3450422622 conn(0x7fd7380772d0 msgr2=0x7fd738077930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:40.748 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 shutdown_connections 2023-12-12T10:08:40.749 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:40.744+0000 7fd73f9b3700 1 -- 172.21.15.42:0/3450422622 wait complete. 2023-12-12T10:08:40.770 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-12-12T10:08:40.784 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T10:08:40.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:40 smithi173 ceph-mon[103308]: pgmap v309: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 717 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:40.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:40 smithi173 ceph-mon[103308]: from='client.25690 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T10:08:40.838 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:40 smithi042 ceph-mon[98894]: pgmap v309: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 717 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:40.839 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:40 smithi042 ceph-mon[98894]: from='client.25690 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T10:08:40.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:40 smithi111 ceph-mon[102565]: pgmap v309: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 717 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:08:40.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:40 smithi111 ceph-mon[102565]: from='client.25690 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T10:08:41.014 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.005+0000 7f8899115700 1 -- 172.21.15.42:0/1332669721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8894101400 msgr2=0x7f8894101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:41.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.005+0000 7f8899115700 1 --2- 172.21.15.42:0/1332669721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8894101400 0x7f8894101880 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8888009990 tx=0x7f8888009ca0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.005+0000 7f8899115700 1 -- 172.21.15.42:0/1332669721 shutdown_connections 2023-12-12T10:08:41.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.005+0000 7f8899115700 1 --2- 172.21.15.42:0/1332669721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8894101dc0 0x7f889410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.005+0000 7f8899115700 1 --2- 172.21.15.42:0/1332669721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88941074a0 0x7f8894107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.005+0000 7f8899115700 1 --2- 172.21.15.42:0/1332669721 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8894101400 0x7f8894101880 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.005+0000 7f8899115700 1 -- 172.21.15.42:0/1332669721 >> 172.21.15.42:0/1332669721 conn(0x7f88940fcf50 msgr2=0x7f88940ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:41.015 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.007+0000 7f8899115700 1 -- 172.21.15.42:0/1332669721 shutdown_connections 2023-12-12T10:08:41.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.012+0000 7f8899115700 1 -- 172.21.15.42:0/1332669721 wait complete. 2023-12-12T10:08:41.016 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.013+0000 7f8899115700 1 Processor -- start 2023-12-12T10:08:41.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.015+0000 7f8899115700 1 -- start start 2023-12-12T10:08:41.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.016+0000 7f8899115700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8894101400 0x7f889419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:41.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.016+0000 7f8899115700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8894101dc0 0x7f889419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:41.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.016+0000 7f8899115700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88941074a0 0x7f889410d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:41.022 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.016+0000 7f8899115700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889410daa0 con 0x7f88941074a0 2023-12-12T10:08:41.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.016+0000 7f8899115700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889410dc10 con 0x7f8894101dc0 2023-12-12T10:08:41.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.016+0000 7f8899115700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889410ddd0 con 0x7f8894101400 2023-12-12T10:08:41.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.016+0000 7f889359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88941074a0 0x7f889410d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:41.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.016+0000 7f889359e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88941074a0 0x7f889410d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37010/0 (socket says 172.21.15.42:37010) 2023-12-12T10:08:41.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.016+0000 7f889359e700 1 -- 172.21.15.42:0/1345468921 learned_addr learned my addr 172.21.15.42:0/1345468921 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:08:41.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.017+0000 7f889359e700 1 -- 172.21.15.42:0/1345468921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8894101400 msgr2=0x7f889419a420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T10:08:41.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.017+0000 7f889359e700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8894101400 0x7f889419a420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.023 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.017+0000 7f889359e700 1 -- 172.21.15.42:0/1345468921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8894101dc0 msgr2=0x7f889419a960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T10:08:41.024 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.017+0000 7f889359e700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8894101dc0 0x7f889419a960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.024 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.017+0000 7f889359e700 1 -- 172.21.15.42:0/1345468921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88880095f0 con 0x7f88941074a0 2023-12-12T10:08:41.024 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.017+0000 7f889359e700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88941074a0 0x7f889410d2d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b970 tx=0x7f888c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:41.024 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.020+0000 7f887bfff700 1 -- 172.21.15.42:0/1345468921 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c00c9b0 con 0x7f88941074a0 2023-12-12T10:08:41.025 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.020+0000 7f887bfff700 1 -- 172.21.15.42:0/1345468921 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f888c014440 con 0x7f88941074a0 2023-12-12T10:08:41.025 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.020+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889410e0b0 con 0x7f88941074a0 2023-12-12T10:08:41.025 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.020+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889419e410 con 0x7f88941074a0 2023-12-12T10:08:41.025 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.020+0000 7f887bfff700 1 -- 172.21.15.42:0/1345468921 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c0075d0 con 0x7f88941074a0 2023-12-12T10:08:41.025 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.021+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8894102b40 con 0x7f88941074a0 2023-12-12T10:08:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.024+0000 7f887bfff700 1 -- 172.21.15.42:0/1345468921 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f888c0145b0 con 0x7f88941074a0 2023-12-12T10:08:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.025+0000 7f887bfff700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f887c0a16b0 0x7f887c0a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:41.029 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.025+0000 7f887bfff700 1 -- 172.21.15.42:0/1345468921 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f888c006e40 con 0x7f88941074a0 2023-12-12T10:08:41.041 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.033+0000 7f8892d9d700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f887c0a16b0 0x7f887c0a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:41.041 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.034+0000 7f8892d9d700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f887c0a16b0 0x7f887c0a3b70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8884009af0 tx=0x7f8884009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:41.042 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.034+0000 7f887bfff700 1 -- 172.21.15.42:0/1345468921 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f888c04f2f0 con 0x7f88941074a0 2023-12-12T10:08:41.417 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.410+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8894196b20 con 0x7f88941074a0 2023-12-12T10:08:41.420 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.417+0000 7f887bfff700 1 -- 172.21.15.42:0/1345468921 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+25310 (secure 0 0 0) 0x7f8894196b20 con 0x7f88941074a0 2023-12-12T10:08:41.423 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T10:08:41.424 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":97,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T10:08:07.704344+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2637927159":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/2112736882":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/3552425799":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/3987075096":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/2637927159":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3046151373":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T10:08:41.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.420+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f887c0a16b0 msgr2=0x7f887c0a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:41.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f887c0a16b0 0x7f887c0a3b70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8884009af0 tx=0x7f8884009260 comp rx=0 tx=0).stop 2023-12-12T10:08:41.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88941074a0 msgr2=0x7f889410d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:41.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88941074a0 0x7f889410d2d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f888c00b970 tx=0x7f888c00bd30 comp rx=0 tx=0).stop 2023-12-12T10:08:41.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 shutdown_connections 2023-12-12T10:08:41.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f88941074a0 0x7f889410d2d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8894101400 0x7f889419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.425 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f887c0a16b0 0x7f887c0a3b70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 --2- 172.21.15.42:0/1345468921 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8894101dc0 0x7f889419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 >> 172.21.15.42:0/1345468921 conn(0x7f88940fcf50 msgr2=0x7f8894073880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 shutdown_connections 2023-12-12T10:08:41.426 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.421+0000 7f8899115700 1 -- 172.21.15.42:0/1345468921 wait complete. 2023-12-12T10:08:41.446 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:56:35.884237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:56:37.283813+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:56:40.918865+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T10:08:41.447 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T10:08:41.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.678+0000 7f486d4cf700 1 -- 172.21.15.42:0/4078918723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100ab0 msgr2=0x7f48681061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:41.680 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.678+0000 7f486d4cf700 1 --2- 172.21.15.42:0/4078918723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100ab0 0x7f48681061e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f485c009990 tx=0x7f485c009ca0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.683 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.681+0000 7f486d4cf700 1 -- 172.21.15.42:0/4078918723 shutdown_connections 2023-12-12T10:08:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.681+0000 7f486d4cf700 1 --2- 172.21.15.42:0/4078918723 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4868106910 0x7f486810e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.681+0000 7f486d4cf700 1 --2- 172.21.15.42:0/4078918723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4868100100 0x7f48681004e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.681+0000 7f486d4cf700 1 --2- 172.21.15.42:0/4078918723 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100ab0 0x7f48681061e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.684 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.681+0000 7f486d4cf700 1 -- 172.21.15.42:0/4078918723 >> 172.21.15.42:0/4078918723 conn(0x7f4868075740 msgr2=0x7f4868075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:41.692 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.689+0000 7f486d4cf700 1 -- 172.21.15.42:0/4078918723 shutdown_connections 2023-12-12T10:08:41.694 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.691+0000 7f486d4cf700 1 -- 172.21.15.42:0/4078918723 wait complete. 2023-12-12T10:08:41.695 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.692+0000 7f486d4cf700 1 Processor -- start 2023-12-12T10:08:41.697 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f486d4cf700 1 -- start start 2023-12-12T10:08:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f486d4cf700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100100 0x7f486810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f486d4cf700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4868106910 0x7f486810e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f486d4cf700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486810c210 0x7f486810c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f486d4cf700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486810ce60 con 0x7f4868106910 2023-12-12T10:08:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f486d4cf700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4868113380 con 0x7f486810c210 2023-12-12T10:08:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f486d4cf700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4868113540 con 0x7f4868100100 2023-12-12T10:08:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f4866ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100100 0x7f486810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:41.698 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f4866ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100100 0x7f486810df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:52004/0 (socket says 172.21.15.42:52004) 2023-12-12T10:08:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f4866ffd700 1 -- 172.21.15.42:0/1787790076 learned_addr learned my addr 172.21.15.42:0/1787790076 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:08:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.693+0000 7f48677fe700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486810c210 0x7f486810c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f4866ffd700 1 -- 172.21.15.42:0/1787790076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486810c210 msgr2=0x7f486810c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f4866ffd700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486810c210 0x7f486810c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f4866ffd700 1 -- 172.21.15.42:0/1787790076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4868106910 msgr2=0x7f486810e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f4866ffd700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4868106910 0x7f486810e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f4866ffd700 1 -- 172.21.15.42:0/1787790076 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c0095f0 con 0x7f4868100100 2023-12-12T10:08:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f48677fe700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486810c210 0x7f486810c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:08:41.699 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f4866ffd700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100100 0x7f486810df50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f485800e9a0 tx=0x7f485800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f484ffff700 1 -- 172.21.15.42:0/1787790076 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858009790 con 0x7f4868100100 2023-12-12T10:08:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f484ffff700 1 -- 172.21.15.42:0/1787790076 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f485800cbd0 con 0x7f4868100100 2023-12-12T10:08:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f484ffff700 1 -- 172.21.15.42:0/1787790076 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4858018680 con 0x7f4868100100 2023-12-12T10:08:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4868113820 con 0x7f4868100100 2023-12-12T10:08:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.694+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4868113ce0 con 0x7f4868100100 2023-12-12T10:08:41.700 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.696+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48540052f0 con 0x7f4868100100 2023-12-12T10:08:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.698+0000 7f484ffff700 1 -- 172.21.15.42:0/1787790076 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f485800cd40 con 0x7f4868100100 2023-12-12T10:08:41.701 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.699+0000 7f484ffff700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48500a1670 0x7f48500a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:41.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.699+0000 7f484ffff700 1 -- 172.21.15.42:0/1787790076 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4858014070 con 0x7f4868100100 2023-12-12T10:08:41.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.699+0000 7f48667fc700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48500a1670 0x7f48500a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:41.702 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.700+0000 7f48667fc700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48500a1670 0x7f48500a3b30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f485c00b2e0 tx=0x7f485c005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:41.705 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:41.703+0000 7f484ffff700 1 -- 172.21.15.42:0/1787790076 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f48580965e0 con 0x7f4868100100 2023-12-12T10:08:41.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:41 smithi173 ceph-mon[103308]: from='client.25748 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T10:08:41.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:41 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1345468921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T10:08:41.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:41 smithi042 ceph-mon[98894]: from='client.25748 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T10:08:41.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:41 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1345468921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T10:08:41.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:41 smithi111 ceph-mon[102565]: from='client.25748 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-12T10:08:41.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:41 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1345468921' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T10:08:42.063 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T10:08:42.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.059+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4854005b20 con 0x7f4868100100 2023-12-12T10:08:42.073 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.060+0000 7f484ffff700 1 -- 172.21.15.42:0/1787790076 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v97) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4858099c00 con 0x7f4868100100 2023-12-12T10:08:42.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.064+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48500a1670 msgr2=0x7f48500a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.064+0000 7f486d4cf700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48500a1670 0x7f48500a3b30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f485c00b2e0 tx=0x7f485c005fd0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.064+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100100 msgr2=0x7f486810df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.064+0000 7f486d4cf700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100100 0x7f486810df50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f485800e9a0 tx=0x7f485800ecb0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.069+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 shutdown_connections 2023-12-12T10:08:42.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.069+0000 7f486d4cf700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f48500a1670 0x7f48500a3b30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.069+0000 7f486d4cf700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486810c210 0x7f486810c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.074 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.069+0000 7f486d4cf700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4868100100 0x7f486810df50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.075 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.069+0000 7f486d4cf700 1 --2- 172.21.15.42:0/1787790076 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4868106910 0x7f486810e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.075 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.069+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 >> 172.21.15.42:0/1787790076 conn(0x7f4868075740 msgr2=0x7f48680fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:42.075 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.070+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 shutdown_connections 2023-12-12T10:08:42.075 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.070+0000 7f486d4cf700 1 -- 172.21.15.42:0/1787790076 wait complete. 2023-12-12T10:08:42.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T10:08:42.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.341+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3285771154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fdc100400 msgr2=0x7f3fdc100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.341+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3285771154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fdc100400 0x7f3fdc100880 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009990 tx=0x7f3fc8009ca0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.341+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3285771154 shutdown_connections 2023-12-12T10:08:42.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.341+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3285771154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc100dc0 0x7f3fdc10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.341+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3285771154 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fdc106490 0x7f3fdc106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.341+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3285771154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fdc100400 0x7f3fdc100880 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.341+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3285771154 >> 172.21.15.42:0/3285771154 conn(0x7f3fdc077560 msgr2=0x7f3fdc077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.342+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3285771154 shutdown_connections 2023-12-12T10:08:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.342+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3285771154 wait complete. 2023-12-12T10:08:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.343+0000 7f3fe2aed700 1 Processor -- start 2023-12-12T10:08:42.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.343+0000 7f3fe2aed700 1 -- start start 2023-12-12T10:08:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fe2aed700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fdc100400 0x7f3fdc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fe2aed700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc100dc0 0x7f3fdc197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fe2aed700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fdc106490 0x7f3fdc19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fe2aed700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19c930 con 0x7f3fdc106490 2023-12-12T10:08:42.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fe2aed700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19caa0 con 0x7f3fdc100dc0 2023-12-12T10:08:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fe2aed700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc19cc60 con 0x7f3fdc100400 2023-12-12T10:08:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fdbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc100dc0 0x7f3fdc197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fdbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc100dc0 0x7f3fdc197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:40094/0 (socket says 172.21.15.42:40094) 2023-12-12T10:08:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fdbfff700 1 -- 172.21.15.42:0/3028064214 learned_addr learned my addr 172.21.15.42:0/3028064214 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:08:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fe0889700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fdc100400 0x7f3fdc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fe108a700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fdc106490 0x7f3fdc19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fdbfff700 1 -- 172.21.15.42:0/3028064214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fdc100400 msgr2=0x7f3fdc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fdbfff700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fdc100400 0x7f3fdc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fdbfff700 1 -- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fdc106490 msgr2=0x7f3fdc19c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.349 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fdbfff700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fdc106490 0x7f3fdc19c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.349 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.344+0000 7f3fdbfff700 1 -- 172.21.15.42:0/3028064214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc80095f0 con 0x7f3fdc100dc0 2023-12-12T10:08:42.349 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.345+0000 7f3fdbfff700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc100dc0 0x7f3fdc197fe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800b2e0 tx=0x7f3fc8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:42.349 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.346+0000 7f3fd9ffb700 1 -- 172.21.15.42:0/3028064214 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc801d070 con 0x7f3fdc100dc0 2023-12-12T10:08:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.346+0000 7f3fd9ffb700 1 -- 172.21.15.42:0/3028064214 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc8021410 con 0x7f3fdc100dc0 2023-12-12T10:08:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.346+0000 7f3fd9ffb700 1 -- 172.21.15.42:0/3028064214 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc8017650 con 0x7f3fdc100dc0 2023-12-12T10:08:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.346+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc19cee0 con 0x7f3fdc100dc0 2023-12-12T10:08:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.346+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc101f80 con 0x7f3fdc100dc0 2023-12-12T10:08:42.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.347+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc101b30 con 0x7f3fdc100dc0 2023-12-12T10:08:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.350+0000 7f3fd9ffb700 1 -- 172.21.15.42:0/3028064214 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f3fc800fa50 con 0x7f3fdc100dc0 2023-12-12T10:08:42.353 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.351+0000 7f3fd9ffb700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f3fcc0a1740 0x7f3fcc0a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.351+0000 7f3fd9ffb700 1 -- 172.21.15.42:0/3028064214 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f3fc80ceb50 con 0x7f3fdc100dc0 2023-12-12T10:08:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.351+0000 7f3fe0889700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f3fcc0a1740 0x7f3fcc0a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:42.354 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.351+0000 7f3fe0889700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f3fcc0a1740 0x7f3fcc0a3c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009c60 tx=0x7f3fd0009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:42.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.353+0000 7f3fd9ffb700 1 -- 172.21.15.42:0/3028064214 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f3fc8097b80 con 0x7f3fdc100dc0 2023-12-12T10:08:42.713 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.710+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3fdc198d50 con 0x7f3fdc100dc0 2023-12-12T10:08:42.714 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.712+0000 7f3fd9ffb700 1 -- 172.21.15.42:0/3028064214 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3fc8026020 con 0x7f3fdc100dc0 2023-12-12T10:08:42.716 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T10:08:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.717+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f3fcc0a1740 msgr2=0x7f3fcc0a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.717+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f3fcc0a1740 0x7f3fcc0a3c00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3fd0009c60 tx=0x7f3fd0009260 comp rx=0 tx=0).stop 2023-12-12T10:08:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.717+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc100dc0 msgr2=0x7f3fdc197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.720 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.717+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc100dc0 0x7f3fdc197fe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800b2e0 tx=0x7f3fc8005740 comp rx=0 tx=0).stop 2023-12-12T10:08:42.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.719+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 shutdown_connections 2023-12-12T10:08:42.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.719+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3fdc106490 0x7f3fdc19c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.719+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3fdc100400 0x7f3fdc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.719+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f3fcc0a1740 0x7f3fcc0a3c00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.719+0000 7f3fe2aed700 1 --2- 172.21.15.42:0/3028064214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3fdc100dc0 0x7f3fdc197fe0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.722 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.719+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 >> 172.21.15.42:0/3028064214 conn(0x7f3fdc077560 msgr2=0x7f3fdc104370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:42.727 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.724+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 shutdown_connections 2023-12-12T10:08:42.727 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.725+0000 7f3fe2aed700 1 -- 172.21.15.42:0/3028064214 wait complete. 2023-12-12T10:08:42.751 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T10:08:42.910 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:42 smithi042 ceph-mon[98894]: pgmap v310: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 717 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:42.911 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:42 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1787790076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T10:08:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.978+0000 7f4e9d911700 1 -- 172.21.15.42:0/1392994659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100420 msgr2=0x7f4e981008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.978+0000 7f4e9d911700 1 --2- 172.21.15.42:0/1392994659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100420 0x7f4e981008a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4e88009990 tx=0x7f4e88009ca0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.978+0000 7f4e9d911700 1 -- 172.21.15.42:0/1392994659 shutdown_connections 2023-12-12T10:08:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.978+0000 7f4e9d911700 1 --2- 172.21.15.42:0/1392994659 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e98100de0 0x7f4e9810d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.978+0000 7f4e9d911700 1 --2- 172.21.15.42:0/1392994659 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e981064a0 0x7f4e98106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.989 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.978+0000 7f4e9d911700 1 --2- 172.21.15.42:0/1392994659 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100420 0x7f4e981008a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.978+0000 7f4e9d911700 1 -- 172.21.15.42:0/1392994659 >> 172.21.15.42:0/1392994659 conn(0x7f4e98077570 msgr2=0x7f4e98077980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.978+0000 7f4e9d911700 1 -- 172.21.15.42:0/1392994659 shutdown_connections 2023-12-12T10:08:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.978+0000 7f4e9d911700 1 -- 172.21.15.42:0/1392994659 wait complete. 2023-12-12T10:08:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.979+0000 7f4e9d911700 1 Processor -- start 2023-12-12T10:08:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.979+0000 7f4e9d911700 1 -- start start 2023-12-12T10:08:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e9d911700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e98100420 0x7f4e98072c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:42.990 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e9d911700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100de0 0x7f4e98073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e9d911700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e981064a0 0x7f4e9806bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e9d911700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9806c6c0 con 0x7f4e98100de0 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e9d911700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9806c830 con 0x7f4e981064a0 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e9d911700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9806c9f0 con 0x7f4e98100420 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e967fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100de0 0x7f4e98073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e967fc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100de0 0x7f4e98073210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37048/0 (socket says 172.21.15.42:37048) 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e96ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e98100420 0x7f4e98072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e967fc700 1 -- 172.21.15.42:0/3750785023 learned_addr learned my addr 172.21.15.42:0/3750785023 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e977fe700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e981064a0 0x7f4e9806bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:42.991 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e967fc700 1 -- 172.21.15.42:0/3750785023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e98100420 msgr2=0x7f4e98072c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e967fc700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e98100420 0x7f4e98072c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e967fc700 1 -- 172.21.15.42:0/3750785023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e981064a0 msgr2=0x7f4e9806bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e967fc700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e981064a0 0x7f4e9806bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.980+0000 7f4e967fc700 1 -- 172.21.15.42:0/3750785023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e880095f0 con 0x7f4e98100de0 2023-12-12T10:08:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.981+0000 7f4e967fc700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100de0 0x7f4e98073210 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000c00 tx=0x7f4e88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.981+0000 7f4e9c90f700 1 -- 172.21.15.42:0/3750785023 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8801d070 con 0x7f4e98100de0 2023-12-12T10:08:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.981+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e98070820 con 0x7f4e98100de0 2023-12-12T10:08:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.981+0000 7f4e9c90f700 1 -- 172.21.15.42:0/3750785023 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4e88022470 con 0x7f4e98100de0 2023-12-12T10:08:42.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.981+0000 7f4e9c90f700 1 -- 172.21.15.42:0/3750785023 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e88017650 con 0x7f4e98100de0 2023-12-12T10:08:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.981+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e98070d50 con 0x7f4e98100de0 2023-12-12T10:08:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.985+0000 7f4e9c90f700 1 -- 172.21.15.42:0/3750785023 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f4e880225e0 con 0x7f4e98100de0 2023-12-12T10:08:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.985+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e98101b40 con 0x7f4e98100de0 2023-12-12T10:08:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.985+0000 7f4e9c90f700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e800a15f0 0x7f4e800a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.986+0000 7f4e9c90f700 1 -- 172.21.15.42:0/3750785023 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4e880ce650 con 0x7f4e98100de0 2023-12-12T10:08:42.993 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.986+0000 7f4e96ffd700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e800a15f0 0x7f4e800a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:42.994 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.986+0000 7f4e96ffd700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e800a15f0 0x7f4e800a3ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009be0 tx=0x7f4e8c009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:42.995 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:42.993+0000 7f4e9c90f700 1 -- 172.21.15.42:0/3750785023 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f4e88097600 con 0x7f4e98100de0 2023-12-12T10:08:42.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:42 smithi111 ceph-mon[102565]: pgmap v310: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 717 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:42.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:42 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1787790076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T10:08:43.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:42 smithi173 ceph-mon[103308]: pgmap v310: 129 pgs: 15 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 70 active+clean; 717 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:43.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:42 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1787790076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T10:08:43.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.344+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e9806d240 con 0x7f4e98100de0 2023-12-12T10:08:43.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.345+0000 7f4e9c90f700 1 -- 172.21.15.42:0/3750785023 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e8809ac20 con 0x7f4e98100de0 2023-12-12T10:08:43.348 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2023-12-12T10:08:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e800a15f0 msgr2=0x7f4e800a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e800a15f0 0x7f4e800a3ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c009be0 tx=0x7f4e8c009260 comp rx=0 tx=0).stop 2023-12-12T10:08:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100de0 msgr2=0x7f4e98073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100de0 0x7f4e98073210 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000c00 tx=0x7f4e88004730 comp rx=0 tx=0).stop 2023-12-12T10:08:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 shutdown_connections 2023-12-12T10:08:43.355 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f4e800a15f0 0x7f4e800a3ab0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.356 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4e981064a0 0x7f4e9806bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.356 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e98100420 0x7f4e98072c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.356 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 --2- 172.21.15.42:0/3750785023 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4e98100de0 0x7f4e98073210 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.356 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 >> 172.21.15.42:0/3750785023 conn(0x7f4e98077570 msgr2=0x7f4e980fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:43.356 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 shutdown_connections 2023-12-12T10:08:43.356 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.352+0000 7f4e9d911700 1 -- 172.21.15.42:0/3750785023 wait complete. 2023-12-12T10:08:43.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T10:08:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.594+0000 7f14320a3700 1 -- 172.21.15.42:0/3185778816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c11adb0 msgr2=0x7f142c11d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:43.597 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.594+0000 7f14320a3700 1 --2- 172.21.15.42:0/3185778816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c11adb0 0x7f142c11d1a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f142401c0d0 tx=0x7f142401c3e0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.597+0000 7f14320a3700 1 -- 172.21.15.42:0/3185778816 shutdown_connections 2023-12-12T10:08:43.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.597+0000 7f14320a3700 1 --2- 172.21.15.42:0/3185778816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c11adb0 0x7f142c11d1a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.597+0000 7f14320a3700 1 --2- 172.21.15.42:0/3185778816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f142c10fd60 0x7f142c1085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.597+0000 7f14320a3700 1 --2- 172.21.15.42:0/3185778816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c108b10 0x7f142c108f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.597+0000 7f14320a3700 1 -- 172.21.15.42:0/3185778816 >> 172.21.15.42:0/3185778816 conn(0x7f142c06c610 msgr2=0x7f142c06ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:43.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.605+0000 7f14320a3700 1 -- 172.21.15.42:0/3185778816 shutdown_connections 2023-12-12T10:08:43.608 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.605+0000 7f14320a3700 1 -- 172.21.15.42:0/3185778816 wait complete. 2023-12-12T10:08:43.609 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.606+0000 7f14320a3700 1 Processor -- start 2023-12-12T10:08:43.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f14320a3700 1 -- start start 2023-12-12T10:08:43.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f14320a3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c108b10 0x7f142c116560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:43.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f14320a3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f142c10fd60 0x7f142c116aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:43.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f14320a3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c11adb0 0x7f142c1a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:43.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f14320a3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1a3110 con 0x7f142c10fd60 2023-12-12T10:08:43.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f14320a3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1a3280 con 0x7f142c108b10 2023-12-12T10:08:43.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f14320a3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1a3440 con 0x7f142c11adb0 2023-12-12T10:08:43.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c108b10 0x7f142c116560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:43.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f142bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c11adb0 0x7f142c1a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:43.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f142b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c108b10 0x7f142c116560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:40120/0 (socket says 172.21.15.42:40120) 2023-12-12T10:08:43.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f142b7fe700 1 -- 172.21.15.42:0/273411111 learned_addr learned my addr 172.21.15.42:0/273411111 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:08:43.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f142bfff700 1 -- 172.21.15.42:0/273411111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c108b10 msgr2=0x7f142c116560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:43.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f142bfff700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c108b10 0x7f142c116560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f142bfff700 1 -- 172.21.15.42:0/273411111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f142c10fd60 msgr2=0x7f142c116aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:43.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.607+0000 7f142bfff700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f142c10fd60 0x7f142c116aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.608+0000 7f142bfff700 1 -- 172.21.15.42:0/273411111 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142401ad10 con 0x7f142c11adb0 2023-12-12T10:08:43.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.608+0000 7f142b7fe700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c108b10 0x7f142c116560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:08:43.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.608+0000 7f142bfff700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c11adb0 0x7f142c1a2a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f142401e010 tx=0x7f1424003d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:43.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.608+0000 7f1428ff9700 1 -- 172.21.15.42:0/273411111 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142401f070 con 0x7f142c11adb0 2023-12-12T10:08:43.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.608+0000 7f1428ff9700 1 -- 172.21.15.42:0/273411111 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14240189e0 con 0x7f142c11adb0 2023-12-12T10:08:43.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.608+0000 7f1428ff9700 1 -- 172.21.15.42:0/273411111 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142402b8b0 con 0x7f142c11adb0 2023-12-12T10:08:43.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.608+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c1a36c0 con 0x7f142c11adb0 2023-12-12T10:08:43.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.608+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c1aaca0 con 0x7f142c11adb0 2023-12-12T10:08:43.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.610+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c1a3880 con 0x7f142c11adb0 2023-12-12T10:08:43.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.611+0000 7f1428ff9700 1 -- 172.21.15.42:0/273411111 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f1424005790 con 0x7f142c11adb0 2023-12-12T10:08:43.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.612+0000 7f1428ff9700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f14140a1630 0x7f14140a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:43.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.612+0000 7f1428ff9700 1 -- 172.21.15.42:0/273411111 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f14240df140 con 0x7f142c11adb0 2023-12-12T10:08:43.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.613+0000 7f142b7fe700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f14140a1630 0x7f14140a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:43.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.613+0000 7f142b7fe700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f14140a1630 0x7f14140a3af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f142c11c690 tx=0x7f1420009290 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:43.624 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.621+0000 7f1428ff9700 1 -- 172.21.15.42:0/273411111 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f14240a80f0 con 0x7f142c11adb0 2023-12-12T10:08:43.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:43 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3028064214' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T10:08:43.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:43 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3750785023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T10:08:43.972 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.969+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f142c1176f0 con 0x7f142c11adb0 2023-12-12T10:08:43.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.974+0000 7f1428ff9700 1 -- 172.21.15.42:0/273411111 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v97) v1 ==== 103+0+11 (secure 0 0 0) 0x7f14240ab710 con 0x7f142c11adb0 2023-12-12T10:08:43.977 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T10:08:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.978+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f14140a1630 msgr2=0x7f14140a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.978+0000 7f14320a3700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f14140a1630 0x7f14140a3af0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f142c11c690 tx=0x7f1420009290 comp rx=0 tx=0).stop 2023-12-12T10:08:43.980 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.978+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c11adb0 msgr2=0x7f142c1a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:43.981 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.978+0000 7f14320a3700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c11adb0 0x7f142c1a2a00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f142401e010 tx=0x7f1424003d70 comp rx=0 tx=0).stop 2023-12-12T10:08:43.981 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.979+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 shutdown_connections 2023-12-12T10:08:43.981 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.979+0000 7f14320a3700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f14140a1630 0x7f14140a3af0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.981 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.979+0000 7f14320a3700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f142c108b10 0x7f142c116560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.979+0000 7f14320a3700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f142c11adb0 0x7f142c1a2a00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.979+0000 7f14320a3700 1 --2- 172.21.15.42:0/273411111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f142c10fd60 0x7f142c116aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:43.982 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.979+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 >> 172.21.15.42:0/273411111 conn(0x7f142c06c610 msgr2=0x7f142c11b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:43.988 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.984+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 shutdown_connections 2023-12-12T10:08:43.992 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:43.988+0000 7f14320a3700 1 -- 172.21.15.42:0/273411111 wait complete. 2023-12-12T10:08:43.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:43 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3028064214' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T10:08:43.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:43 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3750785023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T10:08:44.012 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-12-12T10:08:44.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:43 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3028064214' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T10:08:44.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:43 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3750785023' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T10:08:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.205+0000 7fbf2131c700 1 -- 172.21.15.42:0/3273335707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c100f30 msgr2=0x7fbf1c104f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:44.208 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.205+0000 7fbf2131c700 1 --2- 172.21.15.42:0/3273335707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c100f30 0x7fbf1c104f70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009990 tx=0x7fbf10009ca0 comp rx=0 tx=0).stop 2023-12-12T10:08:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.205+0000 7fbf2131c700 1 -- 172.21.15.42:0/3273335707 shutdown_connections 2023-12-12T10:08:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.205+0000 7fbf2131c700 1 --2- 172.21.15.42:0/3273335707 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1c1056a0 0x7fbf1c10d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.205+0000 7fbf2131c700 1 --2- 172.21.15.42:0/3273335707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c100580 0x7fbf1c100960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.205+0000 7fbf2131c700 1 --2- 172.21.15.42:0/3273335707 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c100f30 0x7fbf1c104f70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.205+0000 7fbf2131c700 1 -- 172.21.15.42:0/3273335707 >> 172.21.15.42:0/3273335707 conn(0x7fbf1c0fbe40 msgr2=0x7fbf1c0fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.205+0000 7fbf2131c700 1 -- 172.21.15.42:0/3273335707 shutdown_connections 2023-12-12T10:08:44.209 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.205+0000 7fbf2131c700 1 -- 172.21.15.42:0/3273335707 wait complete. 2023-12-12T10:08:44.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.206+0000 7fbf2131c700 1 Processor -- start 2023-12-12T10:08:44.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.208+0000 7fbf2131c700 1 -- start start 2023-12-12T10:08:44.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.208+0000 7fbf2131c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1c100580 0x7fbf1c10c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:44.219 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf2131c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c1056a0 0x7fbf1c10c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:44.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf2131c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c10cfa0 0x7fbf1c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:44.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf2131c700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c19a940 con 0x7fbf1c1056a0 2023-12-12T10:08:44.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf2131c700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c19aab0 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf2131c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c19ac70 con 0x7fbf1c100580 2023-12-12T10:08:44.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf1b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c10cfa0 0x7fbf1c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:44.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf1b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c10cfa0 0x7fbf1c19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:40132/0 (socket says 172.21.15.42:40132) 2023-12-12T10:08:44.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf1b7fe700 1 -- 172.21.15.42:0/4261012708 learned_addr learned my addr 172.21.15.42:0/4261012708 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:08:44.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf1b7fe700 1 -- 172.21.15.42:0/4261012708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1c100580 msgr2=0x7fbf1c10c3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T10:08:44.220 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf1b7fe700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1c100580 0x7fbf1c10c3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:44.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf1b7fe700 1 -- 172.21.15.42:0/4261012708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c1056a0 msgr2=0x7fbf1c10c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:44.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf1b7fe700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c1056a0 0x7fbf1c10c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:44.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.209+0000 7fbf1b7fe700 1 -- 172.21.15.42:0/4261012708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100095f0 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.210+0000 7fbf1b7fe700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c10cfa0 0x7fbf1c19a170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400ef30 tx=0x7fbf1400c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:44.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.213+0000 7fbf03fff700 1 -- 172.21.15.42:0/4261012708 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14018070 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.213+0000 7fbf03fff700 1 -- 172.21.15.42:0/4261012708 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbf1400f040 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.213+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c19aef0 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.213+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c1a28d0 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.221 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.213+0000 7fbf03fff700 1 -- 172.21.15.42:0/4261012708 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14003de0 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.222 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.218+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf080052f0 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.219+0000 7fbf03fff700 1 -- 172.21.15.42:0/4261012708 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fbf1400caa0 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.220+0000 7fbf03fff700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fbf040a17c0 0x7fbf040a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:08:44.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.220+0000 7fbf03fff700 1 -- 172.21.15.42:0/4261012708 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbf140ccfd0 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.220+0000 7fbf1affd700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fbf040a17c0 0x7fbf040a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:08:44.223 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.220+0000 7fbf1affd700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fbf040a17c0 0x7fbf040a3c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009730 tx=0x7fbf0c006cb0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:08:44.228 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.225+0000 7fbf03fff700 1 -- 172.21.15.42:0/4261012708 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fbf140999b0 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.626+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7fbf08005160 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.641 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.638+0000 7fbf03fff700 1 -- 172.21.15.42:0/4261012708 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+734410 (secure 0 0 0) 0x7fbf14099540 con 0x7fbf1c10cfa0 2023-12-12T10:08:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T10:08:44.652 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T10:08:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 25, 2023-12-12T10:08:44.652 INFO:teuthology.orchestra.run.smithi042.stdout: "active_gid": 15528, 2023-12-12T10:08:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "active_name": "x", 2023-12-12T10:08:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addrs": { 2023-12-12T10:08:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2023-12-12T10:08:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2023-12-12T10:08:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2023-12-12T10:08:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1776845983 2023-12-12T10:08:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2023-12-12T10:08:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2023-12-12T10:08:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1776845983 2023-12-12T10:08:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addr": "172.21.15.42:6801/1776845983", 2023-12-12T10:08:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "active_change": "2023-12-12T09:58:36.825319+0000", 2023-12-12T10:08:44.654 INFO:teuthology.orchestra.run.smithi042.stdout: "active_mgr_features": 4540138322906710015, 2023-12-12T10:08:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2023-12-12T10:08:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "standbys": [ 2023-12-12T10:08:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 15522, 2023-12-12T10:08:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "z", 2023-12-12T10:08:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540138322906710015, 2023-12-12T10:08:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2023-12-12T10:08:44.655 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2023-12-12T10:08:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.657 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.658 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.660 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.665 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.667 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2023-12-12T10:08:44.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2023-12-12T10:08:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2023-12-12T10:08:44.669 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2023-12-12T10:08:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2023-12-12T10:08:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2023-12-12T10:08:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2023-12-12T10:08:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2023-12-12T10:08:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2023-12-12T10:08:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2023-12-12T10:08:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2023-12-12T10:08:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2023-12-12T10:08:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.673 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2023-12-12T10:08:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.674 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2023-12-12T10:08:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2023-12-12T10:08:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2023-12-12T10:08:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.675 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2023-12-12T10:08:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2023-12-12T10:08:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2023-12-12T10:08:44.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2023-12-12T10:08:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.677 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2023-12-12T10:08:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2023-12-12T10:08:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.678 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-12T10:08:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.679 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2023-12-12T10:08:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2023-12-12T10:08:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.680 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2023-12-12T10:08:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2023-12-12T10:08:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2023-12-12T10:08:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2023-12-12T10:08:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.682 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2023-12-12T10:08:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2023-12-12T10:08:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2023-12-12T10:08:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2023-12-12T10:08:44.684 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2023-12-12T10:08:44.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2023-12-12T10:08:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2023-12-12T10:08:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-12T10:08:44.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2023-12-12T10:08:44.687 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2023-12-12T10:08:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-12T10:08:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2023-12-12T10:08:44.688 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2023-12-12T10:08:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2023-12-12T10:08:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-12T10:08:44.689 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-12T10:08:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2023-12-12T10:08:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2023-12-12T10:08:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.690 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2023-12-12T10:08:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2023-12-12T10:08:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2023-12-12T10:08:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2023-12-12T10:08:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-12T10:08:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2023-12-12T10:08:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2023-12-12T10:08:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2023-12-12T10:08:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.692 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2023-12-12T10:08:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-12T10:08:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2023-12-12T10:08:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2023-12-12T10:08:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:44.693 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2023-12-12T10:08:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2023-12-12T10:08:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-12T10:08:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-12T10:08:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.694 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.696 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.700 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.701 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2023-12-12T10:08:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2023-12-12T10:08:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:44.703 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-12T10:08:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.704 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2023-12-12T10:08:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2023-12-12T10:08:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2023-12-12T10:08:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.705 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2023-12-12T10:08:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2023-12-12T10:08:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap" 2023-12-12T10:08:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2023-12-12T10:08:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.706 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2023-12-12T10:08:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2023-12-12T10:08:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.707 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-12T10:08:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.708 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-12T10:08:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2023-12-12T10:08:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2023-12-12T10:08:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:44.710 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-12T10:08:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-12T10:08:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2023-12-12T10:08:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2023-12-12T10:08:44.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:44.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:44.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.712 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-12T10:08:44.712 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.713 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.713 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.713 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2023-12-12T10:08:44.713 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:44.713 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:44.713 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.714 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2023-12-12T10:08:44.714 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2023-12-12T10:08:44.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:44.714 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.714 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2023-12-12T10:08:44.714 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.714 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.714 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.715 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-12T10:08:44.715 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.715 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.715 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2023-12-12T10:08:44.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2023-12-12T10:08:44.715 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.715 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2023-12-12T10:08:44.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.716 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.716 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.716 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-12T10:08:44.716 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.716 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.717 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2023-12-12T10:08:44.717 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2023-12-12T10:08:44.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.717 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.717 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2023-12-12T10:08:44.717 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.717 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.718 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.718 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-12T10:08:44.718 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.718 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.718 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2023-12-12T10:08:44.718 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2023-12-12T10:08:44.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.719 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.719 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.719 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.719 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.719 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.719 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-12T10:08:44.719 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-12T10:08:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2023-12-12T10:08:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2023-12-12T10:08:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:44.721 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.721 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.721 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.721 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-12T10:08:44.721 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.721 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.721 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2023-12-12T10:08:44.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2023-12-12T10:08:44.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:44.722 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.722 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2023-12-12T10:08:44.722 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.722 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.722 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.722 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-12T10:08:44.723 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.723 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2023-12-12T10:08:44.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2023-12-12T10:08:44.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2023-12-12T10:08:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2023-12-12T10:08:44.724 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2023-12-12T10:08:44.725 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.725 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-12T10:08:44.725 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.725 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.725 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2023-12-12T10:08:44.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2023-12-12T10:08:44.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.726 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.726 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.726 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.726 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.726 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-12T10:08:44.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.727 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-12T10:08:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2023-12-12T10:08:44.728 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2023-12-12T10:08:44.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.729 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.729 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.729 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.729 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.729 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-12T10:08:44.729 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.730 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.730 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2023-12-12T10:08:44.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2023-12-12T10:08:44.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.730 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.730 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-12T10:08:44.731 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.731 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.731 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.731 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:44.731 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.731 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.731 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2023-12-12T10:08:44.732 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2023-12-12T10:08:44.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.732 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.732 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-12T10:08:44.732 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.732 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.732 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2023-12-12T10:08:44.733 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.733 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2023-12-12T10:08:44.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2023-12-12T10:08:44.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.733 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.734 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-12T10:08:44.734 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.734 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.734 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.734 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2023-12-12T10:08:44.734 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.734 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.735 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2023-12-12T10:08:44.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2023-12-12T10:08:44.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.735 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.735 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-12T10:08:44.735 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.735 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.735 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.737 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-12T10:08:44.737 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.737 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.737 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.737 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2023-12-12T10:08:44.737 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.738 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.738 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2023-12-12T10:08:44.738 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2023-12-12T10:08:44.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.738 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.738 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-12T10:08:44.738 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.739 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.739 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.739 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2023-12-12T10:08:44.739 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.739 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.739 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2023-12-12T10:08:44.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2023-12-12T10:08:44.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.740 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.740 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-12T10:08:44.740 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.740 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2023-12-12T10:08:44.740 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.741 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.741 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2023-12-12T10:08:44.741 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2023-12-12T10:08:44.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.741 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.741 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-12T10:08:44.742 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.742 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.742 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2023-12-12T10:08:44.742 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.742 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.742 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2023-12-12T10:08:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2023-12-12T10:08:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-12T10:08:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.743 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2023-12-12T10:08:44.744 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.744 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.744 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2023-12-12T10:08:44.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2023-12-12T10:08:44.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.744 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.745 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-12T10:08:44.745 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.745 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.745 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2023-12-12T10:08:44.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.746 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2023-12-12T10:08:44.746 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2023-12-12T10:08:44.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.746 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-12T10:08:44.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.747 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:44.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.747 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2023-12-12T10:08:44.747 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2023-12-12T10:08:44.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.747 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.749 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2023-12-12T10:08:44.749 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2023-12-12T10:08:44.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.749 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.749 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-12T10:08:44.749 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.749 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.749 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.750 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:44.750 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.750 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.750 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.750 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2023-12-12T10:08:44.750 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2023-12-12T10:08:44.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.750 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.751 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-12T10:08:44.751 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.751 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.751 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.751 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2023-12-12T10:08:44.751 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.751 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.752 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2023-12-12T10:08:44.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2023-12-12T10:08:44.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.752 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-12T10:08:44.752 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.752 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.753 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2023-12-12T10:08:44.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.753 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.753 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.753 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2023-12-12T10:08:44.753 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2023-12-12T10:08:44.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.754 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.754 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.754 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.754 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.754 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2023-12-12T10:08:44.754 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.755 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.755 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2023-12-12T10:08:44.755 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2023-12-12T10:08:44.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.755 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:44.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.756 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.756 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-12T10:08:44.756 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.756 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.756 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2023-12-12T10:08:44.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2023-12-12T10:08:44.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2023-12-12T10:08:44.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.757 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-12T10:08:44.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.758 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2023-12-12T10:08:44.758 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2023-12-12T10:08:44.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.758 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2023-12-12T10:08:44.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.759 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.759 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-12T10:08:44.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.759 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.760 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2023-12-12T10:08:44.760 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2023-12-12T10:08:44.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.760 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.760 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2023-12-12T10:08:44.760 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.760 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.761 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2023-12-12T10:08:44.761 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.761 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.761 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2023-12-12T10:08:44.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2023-12-12T10:08:44.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.762 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.762 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.762 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.762 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-12T10:08:44.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.763 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.764 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2023-12-12T10:08:44.764 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.764 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.764 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2023-12-12T10:08:44.764 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.765 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2023-12-12T10:08:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2023-12-12T10:08:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.766 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.767 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.767 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.767 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-12T10:08:44.767 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.767 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.767 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.768 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.768 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.768 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.768 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.769 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.769 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.769 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.769 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.769 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.769 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.769 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.769 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.770 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2023-12-12T10:08:44.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2023-12-12T10:08:44.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.770 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.770 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.770 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.770 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.771 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.771 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-12T10:08:44.771 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.771 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.771 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.771 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.772 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.772 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.772 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.772 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.772 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-12T10:08:44.772 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.773 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.773 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.773 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.773 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.773 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.774 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.774 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.774 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.774 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.774 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.774 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.774 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.774 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.774 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.775 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.775 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.775 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.776 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.776 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.776 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.776 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.776 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.776 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.776 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.777 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.777 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-12T10:08:44.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-12T10:08:44.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.777 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.777 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.778 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-12T10:08:44.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.778 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-12T10:08:44.779 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-12T10:08:44.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.779 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.779 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2023-12-12T10:08:44.779 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.779 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.779 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-12T10:08:44.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.780 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2023-12-12T10:08:44.780 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2023-12-12T10:08:44.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.780 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2023-12-12T10:08:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.781 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.782 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2023-12-12T10:08:44.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2023-12-12T10:08:44.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.782 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.782 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:44.782 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.783 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-12T10:08:44.783 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.783 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.783 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2023-12-12T10:08:44.783 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2023-12-12T10:08:44.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.784 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.784 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.784 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.784 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.784 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.784 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2023-12-12T10:08:44.784 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.785 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.785 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2023-12-12T10:08:44.785 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2023-12-12T10:08:44.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2023-12-12T10:08:44.786 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.786 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.786 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.786 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2023-12-12T10:08:44.786 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2023-12-12T10:08:44.786 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.786 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2023-12-12T10:08:44.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.787 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.787 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2023-12-12T10:08:44.787 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2023-12-12T10:08:44.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.787 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-12T10:08:44.787 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.788 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.788 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.788 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-12T10:08:44.788 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.788 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2023-12-12T10:08:44.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2023-12-12T10:08:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-12T10:08:44.789 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.790 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.790 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2023-12-12T10:08:44.790 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2023-12-12T10:08:44.790 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.790 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.790 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.791 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.791 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.791 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.791 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-12T10:08:44.791 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.791 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.791 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2023-12-12T10:08:44.792 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2023-12-12T10:08:44.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.792 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.792 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.792 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.792 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-12T10:08:44.793 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.793 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.793 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2023-12-12T10:08:44.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2023-12-12T10:08:44.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.793 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.794 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.794 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.794 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.794 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-12T10:08:44.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.795 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2023-12-12T10:08:44.795 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2023-12-12T10:08:44.795 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.795 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.795 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.795 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.795 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-12T10:08:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2023-12-12T10:08:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2023-12-12T10:08:44.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.797 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.797 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2023-12-12T10:08:44.797 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.797 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2023-12-12T10:08:44.797 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.798 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.798 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2023-12-12T10:08:44.798 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2023-12-12T10:08:44.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.798 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.798 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.798 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.799 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.799 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.799 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-12T10:08:44.799 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.799 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.799 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2023-12-12T10:08:44.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2023-12-12T10:08:44.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.800 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.800 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.800 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.800 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.800 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.800 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-12T10:08:44.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.801 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.801 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2023-12-12T10:08:44.801 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2023-12-12T10:08:44.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.802 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.802 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.802 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.802 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-12T10:08:44.802 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.802 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.802 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2023-12-12T10:08:44.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2023-12-12T10:08:44.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.803 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.803 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.803 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.803 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.803 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2023-12-12T10:08:44.804 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.804 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.804 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2023-12-12T10:08:44.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2023-12-12T10:08:44.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.804 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.805 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.805 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.805 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.805 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-12T10:08:44.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.805 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.806 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.806 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2023-12-12T10:08:44.806 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2023-12-12T10:08:44.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.806 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.806 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.806 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.806 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.807 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.807 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-12T10:08:44.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.807 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.807 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.807 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.808 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2023-12-12T10:08:44.808 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:44.808 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:44.808 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.808 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.808 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.808 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.809 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.809 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.809 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.809 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.809 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.809 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.809 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.809 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.809 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.810 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.810 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.810 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.810 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.810 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.810 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.811 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.811 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.811 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.811 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.812 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.812 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.812 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.812 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.812 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.813 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.813 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.813 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.813 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.813 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.813 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.813 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.813 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.814 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.814 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.814 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.814 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.815 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.815 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.815 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.815 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.815 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.815 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.816 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.816 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2023-12-12T10:08:44.816 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2023-12-12T10:08:44.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.816 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.816 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2023-12-12T10:08:44.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.817 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.817 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.817 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-12T10:08:44.817 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.817 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.817 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2023-12-12T10:08:44.818 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2023-12-12T10:08:44.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.818 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.818 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2023-12-12T10:08:44.818 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.818 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.818 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-12T10:08:44.819 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.819 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.819 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.819 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.819 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2023-12-12T10:08:44.820 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:44.820 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:44.820 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.820 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-12T10:08:44.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-12T10:08:44.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.820 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.820 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2023-12-12T10:08:44.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-12T10:08:44.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.822 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.822 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.822 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.822 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.822 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.823 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.823 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.823 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-12T10:08:44.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-12T10:08:44.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.824 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.824 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.824 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.824 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.824 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.824 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.825 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-12T10:08:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-12T10:08:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.827 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.827 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.827 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.827 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.828 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2023-12-12T10:08:44.828 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-12T10:08:44.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.828 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.828 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.828 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.829 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.830 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.830 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.830 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.831 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-12T10:08:44.831 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-12T10:08:44.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.831 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.831 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.831 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.831 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.832 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.832 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.832 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.832 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-12T10:08:44.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.834 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-12T10:08:44.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-12T10:08:44.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.834 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.835 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.835 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-12T10:08:44.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-12T10:08:44.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.836 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.836 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.836 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.836 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.837 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.837 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.837 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.837 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-12T10:08:44.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-12T10:08:44.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.838 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.838 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.838 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.838 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.839 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-12T10:08:44.839 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-12T10:08:44.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.839 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.839 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.839 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.840 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.840 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.840 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.840 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.840 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.840 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-12T10:08:44.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.842 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.842 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2023-12-12T10:08:44.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-12T10:08:44.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.842 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.842 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2023-12-12T10:08:44.843 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.843 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.843 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.843 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.843 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.843 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-12T10:08:44.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-12T10:08:44.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.844 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.844 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.844 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.844 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.845 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.845 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.845 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2023-12-12T10:08:44.845 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2023-12-12T10:08:44.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.845 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.846 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.846 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.846 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.846 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.847 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2023-12-12T10:08:44.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-12T10:08:44.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.847 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2023-12-12T10:08:44.847 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.847 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.847 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.848 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.848 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.848 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.848 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-12T10:08:44.848 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-12T10:08:44.848 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.848 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.849 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.849 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.849 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.849 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.849 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.849 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.849 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.850 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-12T10:08:44.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-12T10:08:44.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.850 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.850 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.850 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.851 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.851 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.851 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-12T10:08:44.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-12T10:08:44.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.852 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.852 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.852 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.852 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.852 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.852 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.852 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-12T10:08:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-12T10:08:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.853 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.854 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.854 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.854 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.854 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.854 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2023-12-12T10:08:44.855 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-12T10:08:44.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.855 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.855 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.855 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.856 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.856 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-12T10:08:44.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-12T10:08:44.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.856 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.857 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.857 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.857 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.857 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.857 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.857 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.858 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.858 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-12T10:08:44.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-12T10:08:44.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.858 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.859 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.859 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.859 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.859 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.859 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-12T10:08:44.859 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-12T10:08:44.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.860 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.860 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.861 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.861 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-12T10:08:44.861 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-12T10:08:44.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.861 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.861 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.862 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.862 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.862 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.862 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.862 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.862 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.862 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-12T10:08:44.863 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-12T10:08:44.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.863 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.863 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.863 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.864 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.864 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.864 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-12T10:08:44.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-12T10:08:44.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.864 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.865 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.865 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.865 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.865 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.865 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.865 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.865 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.866 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-12T10:08:44.866 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-12T10:08:44.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.866 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.866 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.866 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.866 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.867 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.867 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.867 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.867 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.867 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.867 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-12T10:08:44.867 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-12T10:08:44.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.868 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.868 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.868 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.868 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.868 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.868 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.868 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.869 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.869 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2023-12-12T10:08:44.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-12T10:08:44.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.869 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.869 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.869 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.870 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.870 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.870 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.870 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-12T10:08:44.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-12T10:08:44.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.871 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.871 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-12T10:08:44.871 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.871 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.871 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.871 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.872 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:44.873 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.873 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.873 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.874 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-12T10:08:44.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-12T10:08:44.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.874 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.874 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2023-12-12T10:08:44.874 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.874 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.874 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.875 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.875 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.875 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-12T10:08:44.875 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-12T10:08:44.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.875 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.876 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2023-12-12T10:08:44.876 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.876 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.876 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.876 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.876 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.877 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2023-12-12T10:08:44.877 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-12T10:08:44.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.877 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.877 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.878 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.878 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.878 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.878 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.878 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.878 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-12T10:08:44.878 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-12T10:08:44.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.879 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.879 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2023-12-12T10:08:44.879 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.879 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.879 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.879 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.880 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.880 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2023-12-12T10:08:44.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-12T10:08:44.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.880 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.880 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.881 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.881 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.881 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.881 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.881 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.881 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2023-12-12T10:08:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-12T10:08:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.883 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.883 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.883 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.883 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2023-12-12T10:08:44.883 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-12T10:08:44.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.883 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.884 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.884 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.884 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.884 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.884 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.884 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.885 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-12T10:08:44.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-12T10:08:44.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.885 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.885 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:44.885 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.885 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.886 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.886 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.886 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.886 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.886 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-12T10:08:44.886 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-12T10:08:44.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.887 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.887 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:44.887 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.887 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.887 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.887 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.887 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.888 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.888 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-12T10:08:44.888 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-12T10:08:44.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.888 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.888 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:44.888 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.889 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.889 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.889 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.889 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.889 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.889 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2023-12-12T10:08:44.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2023-12-12T10:08:44.890 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.890 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.890 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.890 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.890 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.890 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.890 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.890 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.890 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.891 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2023-12-12T10:08:44.891 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2023-12-12T10:08:44.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.891 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.891 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.892 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.892 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.892 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.892 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.892 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.892 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.892 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2023-12-12T10:08:44.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2023-12-12T10:08:44.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.893 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.893 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.893 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.893 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.893 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.893 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2023-12-12T10:08:44.894 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.894 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.894 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2023-12-12T10:08:44.894 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2023-12-12T10:08:44.894 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.894 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.895 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2023-12-12T10:08:44.895 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.895 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.895 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.895 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.895 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.895 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.896 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2023-12-12T10:08:44.896 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2023-12-12T10:08:44.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.896 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.896 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.896 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.896 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.897 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.897 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.897 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.897 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.897 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.898 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.898 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.898 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.898 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.898 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.899 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.899 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.899 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.899 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.899 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.899 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.901 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.901 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.901 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.901 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.901 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.901 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.902 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.902 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.902 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.902 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.902 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.902 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.902 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.902 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.903 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.903 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.903 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.903 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.903 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.903 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.903 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.904 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.904 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.904 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.904 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.904 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.904 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.905 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.905 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.905 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.905 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2023-12-12T10:08:44.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2023-12-12T10:08:44.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.905 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.906 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.906 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.906 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.906 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2023-12-12T10:08:44.906 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.906 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.907 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2023-12-12T10:08:44.907 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2023-12-12T10:08:44.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.907 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.907 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.907 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2023-12-12T10:08:44.908 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2023-12-12T10:08:44.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.909 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.909 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2023-12-12T10:08:44.909 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.909 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.909 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.909 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.909 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.910 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.910 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.910 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2023-12-12T10:08:44.910 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2023-12-12T10:08:44.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.910 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.911 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2023-12-12T10:08:44.911 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.911 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.911 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.911 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.911 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.911 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.911 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.911 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.912 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2023-12-12T10:08:44.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2023-12-12T10:08:44.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.912 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.912 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.912 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.912 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.912 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.913 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.913 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.913 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.913 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.913 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2023-12-12T10:08:44.913 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2023-12-12T10:08:44.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.914 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.914 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.914 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2023-12-12T10:08:44.914 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.914 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.914 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.914 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.914 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.915 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.915 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2023-12-12T10:08:44.915 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2023-12-12T10:08:44.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.915 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.915 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2023-12-12T10:08:44.915 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.916 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.916 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.916 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.916 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2023-12-12T10:08:44.916 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.916 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.916 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.916 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2023-12-12T10:08:44.917 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2023-12-12T10:08:44.918 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.918 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.918 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.918 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.918 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2023-12-12T10:08:44.918 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.920 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.920 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2023-12-12T10:08:44.920 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:44.920 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:44.921 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.921 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2023-12-12T10:08:44.921 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2023-12-12T10:08:44.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.921 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.921 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.921 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.921 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.921 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2023-12-12T10:08:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.922 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.923 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.923 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.923 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.923 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.923 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.923 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.923 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.924 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.924 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.924 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.924 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.924 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.924 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.924 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.924 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.925 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.925 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.925 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.925 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.925 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.925 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.925 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.926 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.926 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.926 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.926 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.926 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.927 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.927 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.927 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.927 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.927 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.927 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.928 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.928 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.928 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.928 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.928 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.928 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.929 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.929 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.929 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.929 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.929 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.930 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.930 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.930 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.930 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2023-12-12T10:08:44.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2023-12-12T10:08:44.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.930 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.931 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2023-12-12T10:08:44.931 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.931 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.931 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.931 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-12T10:08:44.931 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.932 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2023-12-12T10:08:44.932 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2023-12-12T10:08:44.932 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.932 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.932 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2023-12-12T10:08:44.932 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.932 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.933 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.933 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-12T10:08:44.933 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.933 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.933 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.933 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2023-12-12T10:08:44.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2023-12-12T10:08:44.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.934 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.934 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.934 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2023-12-12T10:08:44.934 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.934 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.934 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2023-12-12T10:08:44.934 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.934 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.935 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.935 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2023-12-12T10:08:44.935 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2023-12-12T10:08:44.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.935 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2023-12-12T10:08:44.936 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.936 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.936 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.936 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-12T10:08:44.936 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.936 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2023-12-12T10:08:44.937 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2023-12-12T10:08:44.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.937 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.937 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:44.937 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.937 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-12T10:08:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.939 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:44.939 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.939 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.939 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.939 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2023-12-12T10:08:44.939 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.940 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2023-12-12T10:08:44.940 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2023-12-12T10:08:44.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.940 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2023-12-12T10:08:44.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.941 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-12T10:08:44.941 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.941 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.941 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.941 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.941 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2023-12-12T10:08:44.942 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:44.942 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:44.942 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.942 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.943 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.943 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.943 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.943 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.943 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.943 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.943 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.943 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.943 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.944 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.944 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.944 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.944 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.944 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.944 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.944 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.944 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.945 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.945 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.945 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.945 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.945 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.945 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.946 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.946 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.946 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.946 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.946 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.946 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.947 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.947 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.947 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.947 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.947 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.947 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.947 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.947 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.947 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.948 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.948 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.948 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.948 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.948 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.948 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.949 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.949 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.949 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.949 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.949 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.950 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.950 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.950 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.950 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2023-12-12T10:08:44.950 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2023-12-12T10:08:44.950 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.950 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.951 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2023-12-12T10:08:44.951 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.951 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.951 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.951 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.951 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.951 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.952 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2023-12-12T10:08:44.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2023-12-12T10:08:44.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.952 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2023-12-12T10:08:44.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.952 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.953 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.953 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.953 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.953 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:44.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:44.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.955 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.955 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.955 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.955 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.955 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2023-12-12T10:08:44.955 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2023-12-12T10:08:44.955 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2023-12-12T10:08:44.955 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.956 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2023-12-12T10:08:44.956 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2023-12-12T10:08:44.956 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.956 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2023-12-12T10:08:44.956 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.956 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.956 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-12T10:08:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2023-12-12T10:08:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2023-12-12T10:08:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.957 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.958 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2023-12-12T10:08:44.958 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.958 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.958 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.958 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-12T10:08:44.958 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.958 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.959 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2023-12-12T10:08:44.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2023-12-12T10:08:44.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.959 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.959 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.959 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.959 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.960 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.960 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2023-12-12T10:08:44.960 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.960 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.960 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.960 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.960 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:44.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:44.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:44.961 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.961 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.961 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2023-12-12T10:08:44.961 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2023-12-12T10:08:44.961 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.961 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.961 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-12T10:08:44.961 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.962 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.962 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.962 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.962 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.963 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.963 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.963 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.963 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.963 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.963 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.963 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.963 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.964 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.964 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.964 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.964 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.964 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.964 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.964 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.965 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.965 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.965 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.965 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.965 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.965 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.965 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.966 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.966 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.966 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.966 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.966 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.967 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.967 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.967 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.967 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.967 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.967 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.967 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.967 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.967 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.968 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.968 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.968 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.968 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.968 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.968 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.968 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.969 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.969 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.969 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.969 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.969 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.969 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.969 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.970 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.970 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2023-12-12T10:08:44.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2023-12-12T10:08:44.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.970 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.970 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.970 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.971 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.971 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.971 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.971 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2023-12-12T10:08:44.971 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.971 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.971 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2023-12-12T10:08:44.972 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2023-12-12T10:08:44.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.972 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.972 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.972 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2023-12-12T10:08:44.972 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.972 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.972 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.972 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2023-12-12T10:08:44.973 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.973 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.973 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2023-12-12T10:08:44.973 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2023-12-12T10:08:44.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.973 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.974 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2023-12-12T10:08:44.974 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.974 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.974 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.974 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-12T10:08:44.974 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.974 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2023-12-12T10:08:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2023-12-12T10:08:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:44.975 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2023-12-12T10:08:44.976 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.976 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-12T10:08:44.976 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.976 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.976 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2023-12-12T10:08:44.976 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2023-12-12T10:08:44.976 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.977 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.977 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.977 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.977 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.977 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2023-12-12T10:08:44.977 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.977 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.978 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.978 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2023-12-12T10:08:44.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2023-12-12T10:08:44.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.978 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:44.978 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2023-12-12T10:08:44.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.979 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-12T10:08:44.979 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.979 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.980 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2023-12-12T10:08:44.980 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:44.980 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:44.980 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.980 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.980 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.981 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.981 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.981 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.981 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.981 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.982 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.982 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.982 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.982 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.983 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.983 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.983 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.984 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.984 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.984 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.985 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.985 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.985 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.985 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.985 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.985 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.985 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.986 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.986 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.986 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.986 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.986 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.986 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.986 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.987 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.987 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.987 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.987 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.987 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.988 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.988 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.988 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.988 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.988 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.988 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2023-12-12T10:08:44.989 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:44.989 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:44.989 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:44.989 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:44.989 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:44.989 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.989 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.989 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.989 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:44.990 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.990 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.991 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.992 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.992 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.992 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.992 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.992 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.992 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.992 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.992 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.993 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.993 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:44.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:44.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.994 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.994 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.994 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.994 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.994 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.994 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.994 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:44.995 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:44.996 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:44.996 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:44.996 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:44.996 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:44.996 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:44.996 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.996 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.996 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.997 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:44.997 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:44.997 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:44.997 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:44.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:44.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:44.997 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:44.997 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:44.998 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:44.998 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:44.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:44.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:44.998 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:44.998 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.999 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:44.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:44.999 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:44.999 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2023-12-12T10:08:45.000 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.000 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.000 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.000 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2023-12-12T10:08:45.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2023-12-12T10:08:45.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.001 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2023-12-12T10:08:45.001 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.001 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.001 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.001 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-12T10:08:45.001 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.002 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.002 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2023-12-12T10:08:45.002 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2023-12-12T10:08:45.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.002 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.003 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2023-12-12T10:08:45.003 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.003 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.003 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-12T10:08:45.003 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.004 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.004 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.004 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.004 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.004 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.005 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.005 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.005 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.005 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.005 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.007 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.007 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.007 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.007 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.007 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.007 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.007 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.008 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.008 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.008 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.008 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.008 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.008 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.009 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.009 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.009 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.009 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.010 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.010 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.010 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.010 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.010 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.010 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.010 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.011 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.011 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.011 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.011 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.011 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.011 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.012 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.012 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.012 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.012 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.012 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.012 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.012 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.013 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.013 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.013 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.013 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2023-12-12T10:08:45.013 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.014 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.014 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.014 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2023-12-12T10:08:45.014 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2023-12-12T10:08:45.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.014 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.015 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2023-12-12T10:08:45.015 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.015 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.015 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.015 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2023-12-12T10:08:45.015 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-12T10:08:45.015 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.015 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.016 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.016 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.016 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.016 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.016 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.017 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.017 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.017 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.017 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.017 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.017 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.017 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.018 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.018 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.018 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.018 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.018 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.018 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.019 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.019 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.019 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.019 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.020 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.020 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.020 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.020 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.020 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.020 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.020 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.021 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.021 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.021 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.021 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.021 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.021 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.022 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.022 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.022 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.022 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.022 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.022 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.022 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.023 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.023 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.023 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.024 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.024 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.024 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.024 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.024 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.025 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.025 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.025 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.025 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.025 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.025 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2023-12-12T10:08:45.026 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2023-12-12T10:08:45.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.026 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.026 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.026 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.026 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.027 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.027 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2023-12-12T10:08:45.027 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-12T10:08:45.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.027 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.027 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2023-12-12T10:08:45.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2023-12-12T10:08:45.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.028 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.028 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2023-12-12T10:08:45.028 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.028 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.028 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.028 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2023-12-12T10:08:45.029 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.029 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.029 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.029 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2023-12-12T10:08:45.029 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2023-12-12T10:08:45.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.029 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.030 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2023-12-12T10:08:45.030 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.030 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.030 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.030 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2023-12-12T10:08:45.030 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.031 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.031 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2023-12-12T10:08:45.031 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2023-12-12T10:08:45.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.031 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.032 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.032 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.032 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.032 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.032 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2023-12-12T10:08:45.032 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.032 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.033 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2023-12-12T10:08:45.033 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2023-12-12T10:08:45.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.033 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.033 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2023-12-12T10:08:45.033 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.034 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.034 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.034 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-12T10:08:45.034 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-12T10:08:45.034 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.034 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.035 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.035 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.035 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.035 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2023-12-12T10:08:45.035 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.035 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.035 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.036 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.036 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.036 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.036 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.036 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.037 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.037 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.037 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.037 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.037 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.037 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.037 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.037 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.038 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.038 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.038 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.038 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.038 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.038 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.039 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.039 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.039 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.039 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.040 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.040 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.040 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.040 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.040 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.040 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.041 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.041 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.041 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.041 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.041 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.041 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.042 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.042 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.042 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.042 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.042 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.042 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.042 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.042 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.043 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.043 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.043 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.043 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.044 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.044 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.044 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.044 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.044 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.044 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.044 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.045 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.045 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2023-12-12T10:08:45.045 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.045 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.046 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.046 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.046 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.046 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.046 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.046 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.047 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.047 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.047 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.047 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.047 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.047 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.047 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.047 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.048 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.048 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.048 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.048 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.048 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.049 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.049 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.049 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.049 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.049 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.049 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.050 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.050 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.050 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.050 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.050 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.050 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.051 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.051 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.051 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.051 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.051 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.051 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.052 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.052 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.052 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.052 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.052 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.052 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.052 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.053 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.053 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.053 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.053 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.053 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.054 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.054 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.054 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.054 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.054 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.054 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.055 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.055 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.055 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.055 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2023-12-12T10:08:45.056 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.056 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.056 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.056 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.056 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.056 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.057 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.057 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.057 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.057 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.057 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.057 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.057 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.057 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.058 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.058 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.058 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.058 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.058 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.058 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.059 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.059 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.059 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.059 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.059 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.060 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.060 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.060 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.060 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.060 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.060 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.060 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.061 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.062 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.064 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.065 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2023-12-12T10:08:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.066 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2023-12-12T10:08:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2023-12-12T10:08:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.067 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-12T10:08:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.068 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.069 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.070 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.072 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.073 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.074 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.075 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.076 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2023-12-12T10:08:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2023-12-12T10:08:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2023-12-12T10:08:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2023-12-12T10:08:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2023-12-12T10:08:45.078 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2023-12-12T10:08:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.079 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2023-12-12T10:08:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.080 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.081 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.083 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.084 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.086 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.087 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.088 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.089 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2023-12-12T10:08:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.090 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.091 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.092 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.093 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.094 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.096 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.099 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2023-12-12T10:08:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.102 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.104 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.106 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.107 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.109 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.110 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2023-12-12T10:08:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2023-12-12T10:08:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.112 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2023-12-12T10:08:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2023-12-12T10:08:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2023-12-12T10:08:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-12T10:08:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.113 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2023-12-12T10:08:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.114 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2023-12-12T10:08:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2023-12-12T10:08:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.115 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-12T10:08:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.116 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2023-12-12T10:08:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2023-12-12T10:08:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.117 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.119 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.120 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.122 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.124 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.126 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.127 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2023-12-12T10:08:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2023-12-12T10:08:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2023-12-12T10:08:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.128 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2023-12-12T10:08:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:45.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2023-12-12T10:08:45.130 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2023-12-12T10:08:45.131 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2023-12-12T10:08:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2023-12-12T10:08:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.132 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2023-12-12T10:08:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2023-12-12T10:08:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.134 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-12T10:08:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-12T10:08:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.135 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.136 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.137 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.138 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.139 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.140 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.141 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.142 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.143 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2023-12-12T10:08:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2023-12-12T10:08:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.145 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-12T10:08:45.146 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-12T10:08:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2023-12-12T10:08:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.147 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2023-12-12T10:08:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2023-12-12T10:08:45.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2023-12-12T10:08:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.149 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2023-12-12T10:08:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2023-12-12T10:08:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.150 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2023-12-12T10:08:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-12T10:08:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.151 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2023-12-12T10:08:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2023-12-12T10:08:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.152 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2023-12-12T10:08:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-12T10:08:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.153 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2023-12-12T10:08:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2023-12-12T10:08:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2023-12-12T10:08:45.154 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.155 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2023-12-12T10:08:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2023-12-12T10:08:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2023-12-12T10:08:45.156 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2023-12-12T10:08:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2023-12-12T10:08:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.157 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2023-12-12T10:08:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2023-12-12T10:08:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2023-12-12T10:08:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2023-12-12T10:08:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2023-12-12T10:08:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2023-12-12T10:08:45.160 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.162 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.163 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.164 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.166 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.167 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.169 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2023-12-12T10:08:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2023-12-12T10:08:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.171 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2023-12-12T10:08:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2023-12-12T10:08:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.173 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.174 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2023-12-12T10:08:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2023-12-12T10:08:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.175 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.176 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.177 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2023-12-12T10:08:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2023-12-12T10:08:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2023-12-12T10:08:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.178 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.179 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2023-12-12T10:08:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2023-12-12T10:08:45.180 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.181 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.183 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.185 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.186 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.187 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.189 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.189 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.189 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.189 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.189 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.189 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.190 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.190 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.190 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.190 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.190 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.191 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.191 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.191 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.191 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.191 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.191 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.192 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.192 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.192 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.192 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.192 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.193 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.193 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.193 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.193 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.193 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.193 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.194 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.194 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.194 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.194 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.194 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.194 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.195 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.195 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.195 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.195 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2023-12-12T10:08:45.195 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2023-12-12T10:08:45.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.196 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.196 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.196 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.196 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.196 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.196 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.196 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.196 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.197 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.197 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.197 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2023-12-12T10:08:45.197 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2023-12-12T10:08:45.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.198 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.198 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.198 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.198 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.198 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.198 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.199 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.199 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.199 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.199 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.199 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.199 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2023-12-12T10:08:45.200 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.200 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.200 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.200 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.200 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.200 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.201 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.201 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.201 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.201 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.201 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.201 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.201 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.201 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.202 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.202 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.202 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.202 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.202 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.202 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.203 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.203 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.203 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.203 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.203 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.203 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.204 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.204 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.204 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.204 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.204 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.204 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.204 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.205 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.205 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.205 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.205 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.205 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.205 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.205 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.206 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.206 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.206 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.206 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.206 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.206 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.206 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.207 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.207 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.207 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.207 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.207 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.208 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.208 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.208 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.208 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.208 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.208 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.209 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.209 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.209 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.209 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.209 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.210 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2023-12-12T10:08:45.210 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.210 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.210 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.210 INFO:teuthology.orchestra.run.smithi042.stdout: "drive_group_interval": { 2023-12-12T10:08:45.210 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "drive_group_interval", 2023-12-12T10:08:45.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.210 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.211 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300.0", 2023-12-12T10:08:45.211 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.211 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.211 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-12T10:08:45.211 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.212 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.212 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.212 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.212 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.213 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.213 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.213 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.213 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.213 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.213 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.213 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.214 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.214 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.214 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.214 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.214 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.214 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.215 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.216 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.216 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.216 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.216 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.216 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.216 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.217 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.217 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.217 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.217 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.217 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.218 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.218 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.218 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.218 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.218 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.218 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.218 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.219 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.219 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.219 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.219 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.219 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.220 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.220 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.220 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.220 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.220 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.220 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.221 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.221 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.221 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2023-12-12T10:08:45.221 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2023-12-12T10:08:45.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.222 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.222 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.222 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2023-12-12T10:08:45.222 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.222 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.222 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.222 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-12T10:08:45.222 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.223 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.223 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.223 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.223 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2023-12-12T10:08:45.223 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.224 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.224 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.224 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.224 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.224 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.224 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.225 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.225 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.225 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.225 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.225 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.225 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.225 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.226 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.226 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.226 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.226 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.226 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.226 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.226 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.227 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.227 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.227 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.227 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.227 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.227 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.227 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.228 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.228 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.228 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.228 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.228 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.229 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.229 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.229 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.229 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.229 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.229 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.230 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.230 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.230 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.230 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.230 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.230 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.230 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.230 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.231 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.231 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.231 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.231 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.231 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.231 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.232 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.232 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.232 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.232 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.232 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.232 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.233 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.233 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2023-12-12T10:08:45.233 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2023-12-12T10:08:45.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.233 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.234 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.234 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.234 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.234 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.234 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.234 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.234 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.235 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2023-12-12T10:08:45.235 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2023-12-12T10:08:45.235 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.235 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.235 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.235 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2023-12-12T10:08:45.236 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.236 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.236 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.236 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.236 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.236 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.236 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.237 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2023-12-12T10:08:45.237 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2023-12-12T10:08:45.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.237 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.237 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.237 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.238 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.238 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.238 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.238 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.238 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.238 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.239 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2023-12-12T10:08:45.239 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2023-12-12T10:08:45.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.239 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.239 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.239 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.240 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.240 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.240 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.240 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.240 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.240 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2023-12-12T10:08:45.241 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2023-12-12T10:08:45.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.241 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.241 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.241 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.241 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.241 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.242 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.242 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.242 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.242 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2023-12-12T10:08:45.242 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2023-12-12T10:08:45.242 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.243 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.243 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.243 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.243 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.243 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.243 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.244 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.244 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.244 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2023-12-12T10:08:45.244 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2023-12-12T10:08:45.244 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.245 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.245 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.245 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.245 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.245 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.245 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.246 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.246 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.246 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2023-12-12T10:08:45.246 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2023-12-12T10:08:45.246 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.246 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.247 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.247 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.247 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.247 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.247 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.247 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.247 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.247 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.248 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.248 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2023-12-12T10:08:45.248 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2023-12-12T10:08:45.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.248 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.248 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.249 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:45.249 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2023-12-12T10:08:45.249 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.249 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.249 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.249 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.250 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2023-12-12T10:08:45.250 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2023-12-12T10:08:45.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.250 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.250 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.250 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.251 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.251 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.251 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.251 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.251 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.251 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.252 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2023-12-12T10:08:45.252 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2023-12-12T10:08:45.252 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.252 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.252 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.252 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.252 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.253 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.253 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.253 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.253 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.253 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2023-12-12T10:08:45.253 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2023-12-12T10:08:45.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.254 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.254 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.254 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.254 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.255 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.255 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.255 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.255 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.255 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.256 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.256 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2023-12-12T10:08:45.256 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.256 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.256 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.256 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2023-12-12T10:08:45.256 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2023-12-12T10:08:45.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.257 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.257 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.257 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.257 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.257 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.257 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2023-12-12T10:08:45.257 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.258 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.258 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2023-12-12T10:08:45.258 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2023-12-12T10:08:45.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.258 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.259 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.259 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.259 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.259 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.259 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2023-12-12T10:08:45.259 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.260 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.260 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.260 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.260 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.260 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.261 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.261 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.261 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.261 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.261 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.261 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.261 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.262 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.262 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.262 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.262 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.262 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.262 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.263 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.263 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.263 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.263 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.263 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.263 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.264 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.264 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.264 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.264 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.264 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.264 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.265 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.265 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.265 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.265 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.265 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.266 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.266 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.266 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.266 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.266 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.266 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.266 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.266 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.267 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.267 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.267 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.267 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.267 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.268 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.268 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.268 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.268 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.268 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.269 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.269 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.269 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.269 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2023-12-12T10:08:45.269 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.270 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.270 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.270 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.270 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.271 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.271 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.271 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.271 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.271 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.271 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.271 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.271 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.272 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.272 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.272 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.272 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.272 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.272 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.273 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.273 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.273 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.273 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.273 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.273 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.273 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.274 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.274 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.274 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.274 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.275 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.275 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.275 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.275 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.275 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.276 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.276 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.276 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.276 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.276 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.276 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.276 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.276 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.277 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.277 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.277 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.277 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.277 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.278 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.278 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.278 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.278 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.279 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.279 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.279 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.279 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.280 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2023-12-12T10:08:45.280 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.280 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.280 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.280 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.281 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.281 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.281 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.281 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.282 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.282 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.282 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.282 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.282 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.282 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.282 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.282 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.283 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.283 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.283 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.283 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.284 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.284 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.284 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.284 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.284 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.284 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.284 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.285 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.286 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.286 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.286 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.286 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.286 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.286 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.286 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.286 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.287 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.287 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.287 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.288 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.288 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.288 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.288 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.288 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.288 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.288 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.289 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.289 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.289 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.289 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.289 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.290 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.290 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2023-12-12T10:08:45.290 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.290 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.290 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.291 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2023-12-12T10:08:45.291 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2023-12-12T10:08:45.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.291 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.291 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-12T10:08:45.291 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.291 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.292 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.292 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.292 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.292 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.292 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:45.292 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:45.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.293 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.293 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2023-12-12T10:08:45.293 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.293 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.293 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.293 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.294 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.294 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.294 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.294 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.295 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.295 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.295 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.295 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.295 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.295 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.295 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.296 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.296 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.296 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.296 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.296 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.296 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.296 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.297 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.297 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.297 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.297 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.297 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.297 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.298 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.298 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.298 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.298 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.298 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.298 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.298 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.299 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.299 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.299 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.299 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.299 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.300 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.300 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.300 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.300 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.300 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.300 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.300 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.300 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.301 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.301 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.301 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.301 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.301 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.301 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.301 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.302 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.302 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.302 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.302 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.302 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.302 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.302 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.303 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.303 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.303 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.303 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.303 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2023-12-12T10:08:45.304 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.304 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.304 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.304 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2023-12-12T10:08:45.304 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2023-12-12T10:08:45.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.304 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.305 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.305 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.305 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.305 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.305 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-12T10:08:45.305 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.306 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.306 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2023-12-12T10:08:45.306 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2023-12-12T10:08:45.306 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.306 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.306 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.306 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.307 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.307 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.307 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.307 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-12T10:08:45.307 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.308 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.308 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2023-12-12T10:08:45.308 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2023-12-12T10:08:45.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.308 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.308 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.309 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.309 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.309 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.309 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-12T10:08:45.309 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.309 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.310 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2023-12-12T10:08:45.310 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2023-12-12T10:08:45.310 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.310 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.310 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.310 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.310 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.311 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.311 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.311 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-12T10:08:45.311 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.311 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.311 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.311 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2023-12-12T10:08:45.312 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2023-12-12T10:08:45.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.312 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.312 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.312 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.312 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.312 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-12T10:08:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2023-12-12T10:08:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2023-12-12T10:08:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.313 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.314 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.314 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.314 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.314 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.314 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.314 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2023-12-12T10:08:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2023-12-12T10:08:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.315 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.316 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.316 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.316 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.316 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.316 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.316 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.316 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.317 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.317 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2023-12-12T10:08:45.317 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2023-12-12T10:08:45.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.317 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.317 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-12T10:08:45.318 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.318 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.318 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.318 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.318 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.318 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.318 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.319 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2023-12-12T10:08:45.319 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2023-12-12T10:08:45.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.319 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.319 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.319 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.319 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.320 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.320 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.320 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.320 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.320 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.321 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:45.321 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:45.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.321 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.321 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.321 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2023-12-12T10:08:45.321 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2023-12-12T10:08:45.321 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.322 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.322 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.322 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.322 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.322 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2023-12-12T10:08:45.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2023-12-12T10:08:45.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.323 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.323 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2023-12-12T10:08:45.323 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.323 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.323 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.324 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.324 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.324 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.324 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2023-12-12T10:08:45.324 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2023-12-12T10:08:45.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.325 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.325 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.325 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.325 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.325 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.325 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.325 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.326 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.326 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.326 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.326 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2023-12-12T10:08:45.326 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2023-12-12T10:08:45.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.326 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.327 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.327 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.327 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.327 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.327 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.327 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.328 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.328 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.328 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.328 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.329 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.329 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.329 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.329 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.329 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.329 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.329 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.329 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.330 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.330 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.330 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.330 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.330 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.330 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.331 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.331 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.331 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.331 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.331 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.331 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.332 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.332 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.332 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.332 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.332 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.332 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.332 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.333 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.333 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.333 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.333 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.333 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.333 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.333 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.334 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.334 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.334 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.334 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.334 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.335 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.335 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.335 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.335 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.335 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.336 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.336 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.336 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.336 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.336 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.336 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.336 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.337 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.337 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.337 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2023-12-12T10:08:45.337 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2023-12-12T10:08:45.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.337 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.338 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2023-12-12T10:08:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2023-12-12T10:08:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2023-12-12T10:08:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2023-12-12T10:08:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-12T10:08:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.341 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2023-12-12T10:08:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.344 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.345 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.346 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.347 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.348 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.349 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.350 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.350 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.350 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.350 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.350 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.350 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.350 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.351 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.351 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.351 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.351 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.351 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.352 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.352 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.352 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.352 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.352 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.352 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.353 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.353 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2023-12-12T10:08:45.353 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.353 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.353 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.353 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.353 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.354 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.354 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.354 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.354 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.354 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.354 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.354 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.355 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.355 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.355 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.355 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.355 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.355 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.355 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.355 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.356 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.356 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.356 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.356 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.357 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.357 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.357 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.357 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.357 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.357 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.358 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.358 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.358 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.358 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.358 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.358 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.359 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.359 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.359 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.359 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.359 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.359 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.359 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.360 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.360 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.360 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.360 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.360 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.360 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.360 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.361 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.361 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.361 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.361 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.361 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.362 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.362 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.362 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.362 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.362 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.362 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.362 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2023-12-12T10:08:45.363 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2023-12-12T10:08:45.363 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.363 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.363 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.363 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2023-12-12T10:08:45.363 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.363 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.363 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.364 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-12T10:08:45.364 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.364 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.364 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2023-12-12T10:08:45.364 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2023-12-12T10:08:45.365 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.365 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.365 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.365 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.365 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.365 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.366 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2023-12-12T10:08:45.366 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.366 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.366 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.366 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2023-12-12T10:08:45.366 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2023-12-12T10:08:45.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.367 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.367 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:45.367 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.367 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.367 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.367 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-12T10:08:45.367 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.368 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.368 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.368 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.368 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.368 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2023-12-12T10:08:45.369 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.369 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.369 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.369 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2023-12-12T10:08:45.369 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2023-12-12T10:08:45.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:45.369 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.369 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2023-12-12T10:08:45.370 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.370 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.370 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.370 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.370 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.370 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.371 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.371 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2023-12-12T10:08:45.371 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2023-12-12T10:08:45.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.371 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.371 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.373 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.374 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.374 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.374 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.374 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.374 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.375 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.375 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.375 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.375 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.375 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.375 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.376 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.376 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.376 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.376 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.376 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.376 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.376 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.376 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.377 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.377 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.377 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.377 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.377 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.377 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.377 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.378 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.378 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.378 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.378 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.378 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.379 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.379 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.379 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.379 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.380 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.380 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.380 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.380 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.380 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.380 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.380 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.381 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.381 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.381 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.381 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.381 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.381 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.381 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.382 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.382 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.382 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.382 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.382 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.382 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.383 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.383 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.383 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.383 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.383 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.383 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.383 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2023-12-12T10:08:45.384 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2023-12-12T10:08:45.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.384 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.384 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.384 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.384 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.384 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.385 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.385 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.385 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.385 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.385 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2023-12-12T10:08:45.385 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2023-12-12T10:08:45.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.386 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.386 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.386 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2023-12-12T10:08:45.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.386 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.386 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.386 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.387 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.387 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.387 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.387 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2023-12-12T10:08:45.387 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2023-12-12T10:08:45.387 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.387 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.387 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.388 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-12T10:08:45.388 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.388 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.388 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.388 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.388 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.388 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.389 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.389 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:45.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.389 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.389 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 25226, 2023-12-12T10:08:45.389 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "y", 2023-12-12T10:08:45.390 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540138322906710015, 2023-12-12T10:08:45.390 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2023-12-12T10:08:45.390 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.390 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2023-12-12T10:08:45.390 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.390 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.390 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.390 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:45.391 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:45.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.391 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.391 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.391 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:45.391 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.391 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.391 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.392 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2023-12-12T10:08:45.392 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.392 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.392 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.393 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.393 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.393 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.393 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.393 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.393 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.393 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.394 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.394 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.394 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.394 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.394 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.394 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.394 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.395 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.395 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.395 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.395 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.395 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.395 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.395 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.396 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.396 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.396 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.396 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.396 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.396 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.397 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.397 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.397 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.397 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.398 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.398 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.398 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.398 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.398 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.398 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.398 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.399 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.399 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.399 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.399 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.399 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.400 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.400 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.400 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.400 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.400 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.400 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.401 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.401 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2023-12-12T10:08:45.401 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2023-12-12T10:08:45.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.401 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.402 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.402 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.402 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.402 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2023-12-12T10:08:45.402 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.403 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.403 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2023-12-12T10:08:45.403 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2023-12-12T10:08:45.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.403 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.403 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2023-12-12T10:08:45.404 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.404 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.404 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2023-12-12T10:08:45.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.404 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.405 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2023-12-12T10:08:45.405 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2023-12-12T10:08:45.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.406 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.406 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.406 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.406 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.406 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2023-12-12T10:08:45.406 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.406 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.407 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2023-12-12T10:08:45.407 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2023-12-12T10:08:45.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.407 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.408 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.408 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.408 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.408 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2023-12-12T10:08:45.408 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.408 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.409 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2023-12-12T10:08:45.409 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2023-12-12T10:08:45.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.409 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.409 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.409 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2023-12-12T10:08:45.410 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.410 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.410 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.410 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2023-12-12T10:08:45.410 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.410 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.410 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.411 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2023-12-12T10:08:45.411 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2023-12-12T10:08:45.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.411 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.411 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.411 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.412 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2023-12-12T10:08:45.412 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.412 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.412 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2023-12-12T10:08:45.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2023-12-12T10:08:45.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.413 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.413 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.413 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.413 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.414 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.414 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-12T10:08:45.414 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.414 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.414 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2023-12-12T10:08:45.414 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2023-12-12T10:08:45.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.415 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.415 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.416 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2023-12-12T10:08:45.416 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.416 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.416 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.416 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.417 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.417 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2023-12-12T10:08:45.417 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.417 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.417 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.417 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2023-12-12T10:08:45.417 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2023-12-12T10:08:45.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.418 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.418 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.418 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.418 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.418 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.418 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2023-12-12T10:08:45.419 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.419 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.419 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2023-12-12T10:08:45.419 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2023-12-12T10:08:45.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.420 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.420 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2023-12-12T10:08:45.420 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.420 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.420 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.420 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2023-12-12T10:08:45.420 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-12T10:08:45.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.421 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.421 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2023-12-12T10:08:45.421 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2023-12-12T10:08:45.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:45.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.422 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.422 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:45.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2023-12-12T10:08:45.422 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2023-12-12T10:08:45.422 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.422 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-12T10:08:45.422 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-12T10:08:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2023-12-12T10:08:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2023-12-12T10:08:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:45.423 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2023-12-12T10:08:45.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:45.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2023-12-12T10:08:45.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.424 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-12T10:08:45.424 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.425 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.425 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2023-12-12T10:08:45.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2023-12-12T10:08:45.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.425 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2023-12-12T10:08:45.426 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.426 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.426 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.426 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-12T10:08:45.426 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-12T10:08:45.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.426 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.427 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2023-12-12T10:08:45.427 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2023-12-12T10:08:45.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.427 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.427 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2023-12-12T10:08:45.427 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2023-12-12T10:08:45.428 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2023-12-12T10:08:45.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.428 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2023-12-12T10:08:45.428 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-12T10:08:45.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.428 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.428 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2023-12-12T10:08:45.429 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2023-12-12T10:08:45.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.429 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2023-12-12T10:08:45.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.429 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.430 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2023-12-12T10:08:45.430 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-12T10:08:45.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.430 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.430 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2023-12-12T10:08:45.431 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2023-12-12T10:08:45.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:45.431 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.431 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:45.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2023-12-12T10:08:45.431 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2023-12-12T10:08:45.431 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.432 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-12T10:08:45.432 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-12T10:08:45.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.432 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.432 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.433 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.433 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.433 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.433 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.434 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.434 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.434 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.434 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.434 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.434 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.434 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.434 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.435 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.435 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.435 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.435 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.435 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.436 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.436 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.436 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.436 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.436 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.436 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.437 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.437 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.437 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.437 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.437 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.438 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.438 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.438 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.438 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.438 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.438 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.438 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.438 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.439 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.439 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.439 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.439 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.439 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.439 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.440 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.440 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.440 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.440 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.440 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.440 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.440 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.441 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.441 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.441 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.441 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2023-12-12T10:08:45.441 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2023-12-12T10:08:45.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.442 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.442 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:45.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.442 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.442 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.442 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-12T10:08:45.442 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.443 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.443 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2023-12-12T10:08:45.443 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2023-12-12T10:08:45.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.443 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.444 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2023-12-12T10:08:45.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.444 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.444 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2023-12-12T10:08:45.444 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2023-12-12T10:08:45.444 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap" 2023-12-12T10:08:45.445 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.445 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2023-12-12T10:08:45.445 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.445 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.445 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2023-12-12T10:08:45.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2023-12-12T10:08:45.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.446 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.446 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.446 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.446 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.446 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-12T10:08:45.447 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.447 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:45.447 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:45.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.447 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.448 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:45.448 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.448 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.448 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-12T10:08:45.448 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.448 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.449 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2023-12-12T10:08:45.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2023-12-12T10:08:45.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.449 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.449 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.449 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:45.449 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:45.450 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.450 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.450 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-12T10:08:45.450 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-12T10:08:45.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.450 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.451 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2023-12-12T10:08:45.451 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2023-12-12T10:08:45.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:45.451 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.451 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.451 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.451 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.452 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.452 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-12T10:08:45.452 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.452 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.452 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.452 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.453 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.453 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2023-12-12T10:08:45.453 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.453 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.453 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.454 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2023-12-12T10:08:45.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2023-12-12T10:08:45.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.454 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.454 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2023-12-12T10:08:45.454 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.454 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.455 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-12T10:08:45.455 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.455 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.456 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2023-12-12T10:08:45.456 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2023-12-12T10:08:45.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.456 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.456 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2023-12-12T10:08:45.456 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.456 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.457 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.457 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-12T10:08:45.457 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.457 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.457 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2023-12-12T10:08:45.457 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2023-12-12T10:08:45.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.458 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.458 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2023-12-12T10:08:45.458 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.458 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.458 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.459 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-12T10:08:45.459 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.459 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.459 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2023-12-12T10:08:45.459 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2023-12-12T10:08:45.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.460 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.461 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.461 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.461 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.461 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.461 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-12T10:08:45.462 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-12T10:08:45.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.462 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.462 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2023-12-12T10:08:45.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2023-12-12T10:08:45.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:45.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.463 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.463 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-12T10:08:45.463 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.464 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2023-12-12T10:08:45.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2023-12-12T10:08:45.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.465 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2023-12-12T10:08:45.465 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.465 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.465 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-12T10:08:45.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.465 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.466 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2023-12-12T10:08:45.466 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2023-12-12T10:08:45.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.466 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.466 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.467 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.467 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.467 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.467 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2023-12-12T10:08:45.467 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2023-12-12T10:08:45.467 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2023-12-12T10:08:45.467 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.467 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-12T10:08:45.468 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.468 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2023-12-12T10:08:45.468 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2023-12-12T10:08:45.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.469 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.469 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.469 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.469 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.469 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.469 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-12T10:08:45.469 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.470 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.470 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2023-12-12T10:08:45.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2023-12-12T10:08:45.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.470 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.471 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.471 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.471 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.471 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-12T10:08:45.471 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.472 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.472 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2023-12-12T10:08:45.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2023-12-12T10:08:45.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.472 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.473 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.473 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.473 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.473 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.473 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-12T10:08:45.473 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.473 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.474 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.474 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2023-12-12T10:08:45.474 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2023-12-12T10:08:45.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.474 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.475 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-12T10:08:45.475 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.475 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.475 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.475 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:45.475 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.476 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.476 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2023-12-12T10:08:45.476 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2023-12-12T10:08:45.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.476 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.476 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-12T10:08:45.476 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.477 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.477 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.477 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2023-12-12T10:08:45.477 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.477 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.478 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2023-12-12T10:08:45.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2023-12-12T10:08:45.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.478 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.478 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-12T10:08:45.478 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.478 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.479 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.479 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2023-12-12T10:08:45.479 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.479 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.479 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2023-12-12T10:08:45.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2023-12-12T10:08:45.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.480 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-12T10:08:45.480 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.480 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:45.481 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.481 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.481 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2023-12-12T10:08:45.481 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2023-12-12T10:08:45.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.481 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-12T10:08:45.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.482 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.483 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2023-12-12T10:08:45.483 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.483 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.484 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2023-12-12T10:08:45.484 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2023-12-12T10:08:45.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.484 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.484 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-12T10:08:45.484 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.484 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.485 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.485 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2023-12-12T10:08:45.485 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.485 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.485 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2023-12-12T10:08:45.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2023-12-12T10:08:45.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.486 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.486 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-12T10:08:45.486 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.486 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.487 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.487 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2023-12-12T10:08:45.487 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.487 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.487 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2023-12-12T10:08:45.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2023-12-12T10:08:45.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.488 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-12T10:08:45.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.488 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.489 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2023-12-12T10:08:45.489 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.489 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.489 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2023-12-12T10:08:45.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2023-12-12T10:08:45.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.490 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.490 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-12T10:08:45.490 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.490 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.490 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.491 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2023-12-12T10:08:45.491 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.491 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.491 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.491 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2023-12-12T10:08:45.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2023-12-12T10:08:45.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.492 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.492 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-12T10:08:45.492 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.492 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.492 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2023-12-12T10:08:45.493 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.493 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.493 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2023-12-12T10:08:45.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2023-12-12T10:08:45.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.493 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.494 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-12T10:08:45.494 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.494 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.494 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.494 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:45.494 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.495 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.495 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2023-12-12T10:08:45.495 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2023-12-12T10:08:45.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.495 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.495 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-12T10:08:45.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.496 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.496 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.496 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2023-12-12T10:08:45.496 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.496 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.497 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2023-12-12T10:08:45.497 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2023-12-12T10:08:45.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.497 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.497 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-12T10:08:45.497 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.497 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.498 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:45.498 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.498 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.498 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.498 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.498 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2023-12-12T10:08:45.498 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2023-12-12T10:08:45.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.499 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.499 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-12T10:08:45.499 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.499 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.499 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.499 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2023-12-12T10:08:45.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.500 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.500 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2023-12-12T10:08:45.500 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2023-12-12T10:08:45.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.500 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.501 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-12T10:08:45.501 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.501 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.501 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.501 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2023-12-12T10:08:45.501 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.501 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.502 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.502 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2023-12-12T10:08:45.502 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2023-12-12T10:08:45.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.502 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.502 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.502 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.503 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.503 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.503 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.503 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2023-12-12T10:08:45.503 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.503 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.504 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2023-12-12T10:08:45.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2023-12-12T10:08:45.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.504 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.504 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:45.504 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.504 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.505 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.505 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-12T10:08:45.505 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.505 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.505 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2023-12-12T10:08:45.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2023-12-12T10:08:45.506 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.506 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.506 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2023-12-12T10:08:45.506 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.506 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.506 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.506 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-12T10:08:45.507 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.507 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.507 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.507 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2023-12-12T10:08:45.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2023-12-12T10:08:45.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.508 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2023-12-12T10:08:45.508 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.508 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.508 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.508 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-12T10:08:45.508 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.509 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.509 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.509 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2023-12-12T10:08:45.509 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2023-12-12T10:08:45.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.509 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.509 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2023-12-12T10:08:45.510 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.510 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.510 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.510 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2023-12-12T10:08:45.510 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.510 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.510 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.511 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2023-12-12T10:08:45.511 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2023-12-12T10:08:45.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.511 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.511 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.511 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.511 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.512 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.512 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-12T10:08:45.512 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.512 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.512 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2023-12-12T10:08:45.513 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2023-12-12T10:08:45.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.513 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:45.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.513 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.513 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2023-12-12T10:08:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2023-12-12T10:08:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2023-12-12T10:08:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.515 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:45.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.515 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2023-12-12T10:08:45.515 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.516 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.516 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2023-12-12T10:08:45.516 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2023-12-12T10:08:45.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.516 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.516 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.517 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.517 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.517 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.517 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-12T10:08:45.517 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.517 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.518 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.518 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.518 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.518 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.518 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.519 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.519 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.519 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.519 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.519 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.519 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.519 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.520 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.520 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.520 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.520 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.520 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.520 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2023-12-12T10:08:45.521 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2023-12-12T10:08:45.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.521 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.521 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.521 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.521 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.521 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-12T10:08:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.523 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.523 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.523 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.523 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.523 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.523 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-12T10:08:45.523 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.524 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.524 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.524 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.524 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.525 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.525 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.525 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.525 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.525 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.525 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.525 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.526 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.526 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.526 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.526 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.526 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.527 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.527 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.527 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.527 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.528 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.528 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.528 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.528 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.528 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.528 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.528 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.528 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.529 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.529 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-12T10:08:45.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-12T10:08:45.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.529 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.529 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.530 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.530 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-12T10:08:45.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.530 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.530 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-12T10:08:45.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-12T10:08:45.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.531 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.531 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2023-12-12T10:08:45.531 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.531 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.532 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.532 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-12T10:08:45.532 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.532 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.532 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2023-12-12T10:08:45.532 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2023-12-12T10:08:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2023-12-12T10:08:45.533 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.534 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.534 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2023-12-12T10:08:45.534 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2023-12-12T10:08:45.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.534 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.535 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.535 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.535 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.535 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-12T10:08:45.535 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.536 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.536 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2023-12-12T10:08:45.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2023-12-12T10:08:45.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.536 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.536 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.537 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.537 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.537 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.537 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2023-12-12T10:08:45.537 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.537 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.538 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2023-12-12T10:08:45.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2023-12-12T10:08:45.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.538 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.538 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2023-12-12T10:08:45.539 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.539 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.539 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.539 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2023-12-12T10:08:45.539 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2023-12-12T10:08:45.539 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.539 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2023-12-12T10:08:45.539 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.540 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2023-12-12T10:08:45.540 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2023-12-12T10:08:45.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.540 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.541 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-12T10:08:45.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.541 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.541 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.541 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-12T10:08:45.541 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.542 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.542 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2023-12-12T10:08:45.542 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2023-12-12T10:08:45.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.542 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.543 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.543 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.543 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.543 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-12T10:08:45.543 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.544 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.544 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2023-12-12T10:08:45.544 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2023-12-12T10:08:45.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.544 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.545 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.545 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.545 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.545 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-12T10:08:45.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.546 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2023-12-12T10:08:45.546 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2023-12-12T10:08:45.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.546 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.547 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.547 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.547 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.547 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.547 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-12T10:08:45.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.548 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.548 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2023-12-12T10:08:45.548 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2023-12-12T10:08:45.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.548 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.548 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.549 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.549 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.549 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.549 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-12T10:08:45.549 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.550 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2023-12-12T10:08:45.550 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2023-12-12T10:08:45.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.550 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.550 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.550 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.551 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.551 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.551 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-12T10:08:45.551 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.551 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.551 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2023-12-12T10:08:45.552 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2023-12-12T10:08:45.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.552 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.552 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2023-12-12T10:08:45.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.553 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.553 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2023-12-12T10:08:45.553 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.553 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.553 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2023-12-12T10:08:45.554 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2023-12-12T10:08:45.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.554 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.554 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.554 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.554 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.554 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-12T10:08:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2023-12-12T10:08:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2023-12-12T10:08:45.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.556 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.556 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.556 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.556 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.556 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.556 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-12T10:08:45.556 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.557 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.557 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2023-12-12T10:08:45.557 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2023-12-12T10:08:45.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.557 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.558 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.558 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.558 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.558 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-12T10:08:45.558 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.559 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.559 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2023-12-12T10:08:45.559 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2023-12-12T10:08:45.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.559 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.560 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.560 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.560 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.561 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.561 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2023-12-12T10:08:45.561 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.561 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.561 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2023-12-12T10:08:45.562 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2023-12-12T10:08:45.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.562 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.562 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.562 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.562 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.562 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.563 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-12T10:08:45.563 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.563 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.563 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2023-12-12T10:08:45.563 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2023-12-12T10:08:45.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.564 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.564 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.564 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.564 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.564 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.565 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-12T10:08:45.565 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.565 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2023-12-12T10:08:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.567 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.567 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.567 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.567 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.567 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.568 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.568 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.568 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.568 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.569 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.569 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.569 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.569 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.569 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.570 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.570 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.571 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.572 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.572 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.572 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.572 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.573 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.573 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.573 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.574 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.574 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.574 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.575 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.575 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.575 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.575 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.575 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.575 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.576 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.576 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.576 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.576 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.576 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.576 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.576 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.577 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.577 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.577 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.577 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.577 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.577 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.578 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.578 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.578 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.578 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.578 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.579 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2023-12-12T10:08:45.579 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2023-12-12T10:08:45.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.579 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.579 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2023-12-12T10:08:45.579 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.579 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.580 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.580 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-12T10:08:45.580 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.580 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.580 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2023-12-12T10:08:45.580 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2023-12-12T10:08:45.581 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.581 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.581 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.581 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2023-12-12T10:08:45.581 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.581 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.581 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.582 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-12T10:08:45.582 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.582 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.582 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.582 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2023-12-12T10:08:45.583 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.583 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.583 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.583 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-12T10:08:45.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-12T10:08:45.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.583 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.585 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.585 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2023-12-12T10:08:45.585 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-12T10:08:45.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.585 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.585 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.586 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.586 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.586 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.586 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.586 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.586 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.587 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-12T10:08:45.587 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-12T10:08:45.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.587 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.587 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.587 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.588 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.588 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.588 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.588 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.588 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.588 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.589 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2023-12-12T10:08:45.589 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2023-12-12T10:08:45.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.589 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.589 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.589 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.589 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.590 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.590 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.590 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.590 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.590 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.590 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-12T10:08:45.591 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-12T10:08:45.591 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.591 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.591 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.591 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.591 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.591 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.592 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.592 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.592 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.592 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2023-12-12T10:08:45.592 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-12T10:08:45.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.593 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.593 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.593 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.593 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.593 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.593 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.593 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.593 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.594 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.594 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-12T10:08:45.594 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-12T10:08:45.594 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.594 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.595 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.595 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.595 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.595 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.595 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.595 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.595 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.596 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-12T10:08:45.596 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-12T10:08:45.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.596 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.596 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.596 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.596 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.597 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.597 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.597 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.597 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.597 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.597 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.597 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-12T10:08:45.597 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-12T10:08:45.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.598 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.598 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.598 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.598 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.598 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.599 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.599 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.599 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.599 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-12T10:08:45.599 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-12T10:08:45.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.599 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.600 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.600 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.600 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.600 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.600 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.601 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.601 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.601 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-12T10:08:45.601 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-12T10:08:45.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.601 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.601 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.602 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.602 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.602 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.602 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.602 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.602 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.603 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-12T10:08:45.603 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-12T10:08:45.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.603 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.603 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.603 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.603 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.604 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.604 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.604 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.604 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.604 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-12T10:08:45.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-12T10:08:45.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.605 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.605 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.605 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.605 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.605 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.605 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.606 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.606 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.606 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-12T10:08:45.606 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-12T10:08:45.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.607 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.607 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.607 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.607 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.607 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.607 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.607 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.608 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.608 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2023-12-12T10:08:45.608 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-12T10:08:45.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.608 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.609 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2023-12-12T10:08:45.609 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.609 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.609 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.609 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.610 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.610 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-12T10:08:45.610 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-12T10:08:45.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.610 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.611 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.611 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.611 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.611 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.611 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.611 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.612 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2023-12-12T10:08:45.612 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2023-12-12T10:08:45.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.612 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.612 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.613 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.613 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.613 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.613 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.613 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.613 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.614 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2023-12-12T10:08:45.614 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-12T10:08:45.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.614 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.614 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2023-12-12T10:08:45.614 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.614 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.615 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.615 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.615 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.615 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-12T10:08:45.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-12T10:08:45.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.616 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.616 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.616 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.617 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.617 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.617 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.617 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-12T10:08:45.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-12T10:08:45.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.618 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.618 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.618 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.618 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.618 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.618 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.619 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-12T10:08:45.619 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-12T10:08:45.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.619 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.620 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.620 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.620 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.620 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.620 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.620 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.620 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.620 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.621 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-12T10:08:45.621 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-12T10:08:45.621 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.621 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.621 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.621 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.621 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.622 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.622 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.622 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.622 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.622 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.622 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2023-12-12T10:08:45.623 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-12T10:08:45.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.623 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.623 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.623 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.623 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.624 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-12T10:08:45.624 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-12T10:08:45.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.625 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.625 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.625 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.625 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.625 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.625 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.625 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.626 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.626 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.626 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-12T10:08:45.626 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-12T10:08:45.626 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.626 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.627 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.627 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.627 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.627 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.627 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.627 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.628 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.628 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-12T10:08:45.628 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-12T10:08:45.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.628 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.628 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.629 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.629 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.629 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.629 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.629 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.629 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.630 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-12T10:08:45.630 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-12T10:08:45.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.630 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.630 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.630 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.631 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.631 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.631 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.631 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.631 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.631 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.631 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-12T10:08:45.632 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-12T10:08:45.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.632 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.632 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.632 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.632 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.632 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.633 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.633 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.633 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.633 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.633 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-12T10:08:45.634 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-12T10:08:45.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.634 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.634 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.634 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.634 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.634 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.635 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.635 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.635 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.635 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-12T10:08:45.635 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-12T10:08:45.635 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.636 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.636 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.636 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.636 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.636 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.636 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.636 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.637 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.637 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-12T10:08:45.637 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-12T10:08:45.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.637 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.638 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.638 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.638 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.638 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.638 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.638 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.639 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2023-12-12T10:08:45.639 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-12T10:08:45.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.639 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.639 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.640 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.640 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.640 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.640 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.640 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.640 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-12T10:08:45.641 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-12T10:08:45.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.641 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.641 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.641 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-12T10:08:45.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.641 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.642 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.642 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.642 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.642 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.642 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.642 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-12T10:08:45.642 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-12T10:08:45.643 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.643 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.643 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.643 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.643 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.643 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.643 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.644 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.644 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.644 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-12T10:08:45.644 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-12T10:08:45.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.645 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.645 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2023-12-12T10:08:45.645 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.645 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.645 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.645 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.645 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.646 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.646 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-12T10:08:45.646 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-12T10:08:45.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.646 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.647 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2023-12-12T10:08:45.647 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.647 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.647 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.647 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.647 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.647 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.648 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2023-12-12T10:08:45.648 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-12T10:08:45.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.648 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.649 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.649 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.649 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.649 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.649 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-12T10:08:45.650 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-12T10:08:45.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.650 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.650 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2023-12-12T10:08:45.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.650 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.650 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.651 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.651 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.651 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.651 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2023-12-12T10:08:45.651 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-12T10:08:45.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.652 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.652 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.652 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.652 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.653 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.653 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.653 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2023-12-12T10:08:45.653 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-12T10:08:45.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.654 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.654 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.654 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.654 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.654 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.655 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.655 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.655 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2023-12-12T10:08:45.655 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-12T10:08:45.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.656 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.656 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.656 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.656 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.656 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.656 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.657 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.657 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-12T10:08:45.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-12T10:08:45.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.657 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.658 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.658 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:45.658 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.658 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.658 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.659 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.659 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.659 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-12T10:08:45.659 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-12T10:08:45.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.660 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-12T10:08:45.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-12T10:08:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.662 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2023-12-12T10:08:45.663 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2023-12-12T10:08:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.664 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2023-12-12T10:08:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2023-12-12T10:08:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.665 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.666 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2023-12-12T10:08:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2023-12-12T10:08:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2023-12-12T10:08:45.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2023-12-12T10:08:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2023-12-12T10:08:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.669 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2023-12-12T10:08:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.670 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2023-12-12T10:08:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2023-12-12T10:08:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.671 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.672 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.673 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.674 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.676 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.677 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.678 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.679 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.680 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.681 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2023-12-12T10:08:45.682 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2023-12-12T10:08:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.683 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2023-12-12T10:08:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2023-12-12T10:08:45.684 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2023-12-12T10:08:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.685 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.686 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2023-12-12T10:08:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2023-12-12T10:08:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.687 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2023-12-12T10:08:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.688 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2023-12-12T10:08:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2023-12-12T10:08:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.689 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2023-12-12T10:08:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.690 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2023-12-12T10:08:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2023-12-12T10:08:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.691 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.692 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2023-12-12T10:08:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2023-12-12T10:08:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2023-12-12T10:08:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.693 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2023-12-12T10:08:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2023-12-12T10:08:45.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2023-12-12T10:08:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.695 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2023-12-12T10:08:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2023-12-12T10:08:45.696 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2023-12-12T10:08:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2023-12-12T10:08:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2023-12-12T10:08:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2023-12-12T10:08:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.697 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2023-12-12T10:08:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2023-12-12T10:08:45.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.699 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2023-12-12T10:08:45.700 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2023-12-12T10:08:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2023-12-12T10:08:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.701 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2023-12-12T10:08:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.702 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.704 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.705 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.708 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.709 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.710 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2023-12-12T10:08:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2023-12-12T10:08:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2023-12-12T10:08:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-12T10:08:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2023-12-12T10:08:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2023-12-12T10:08:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2023-12-12T10:08:45.714 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-12T10:08:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2023-12-12T10:08:45.715 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2023-12-12T10:08:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2023-12-12T10:08:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.716 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2023-12-12T10:08:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2023-12-12T10:08:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2023-12-12T10:08:45.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2023-12-12T10:08:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-12T10:08:45.718 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2023-12-12T10:08:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2023-12-12T10:08:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-12T10:08:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.720 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2023-12-12T10:08:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.722 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2023-12-12T10:08:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2023-12-12T10:08:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.723 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2023-12-12T10:08:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-12T10:08:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.724 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2023-12-12T10:08:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.725 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.726 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.727 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.729 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.731 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.734 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2023-12-12T10:08:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2023-12-12T10:08:45.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2023-12-12T10:08:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2023-12-12T10:08:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2023-12-12T10:08:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2023-12-12T10:08:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.740 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2023-12-12T10:08:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2023-12-12T10:08:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2023-12-12T10:08:45.741 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2023-12-12T10:08:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2023-12-12T10:08:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2023-12-12T10:08:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-12T10:08:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.743 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2023-12-12T10:08:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2023-12-12T10:08:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2023-12-12T10:08:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.744 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-12T10:08:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2023-12-12T10:08:45.745 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2023-12-12T10:08:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.746 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2023-12-12T10:08:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:45.747 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2023-12-12T10:08:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2023-12-12T10:08:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-12T10:08:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.750 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.751 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.755 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.756 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.756 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.756 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.756 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.756 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.756 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.757 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.757 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.758 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.758 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.758 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.758 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.759 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2023-12-12T10:08:45.759 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2023-12-12T10:08:45.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.760 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.760 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2023-12-12T10:08:45.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.761 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2023-12-12T10:08:45.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2023-12-12T10:08:45.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.761 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2023-12-12T10:08:45.761 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.761 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2023-12-12T10:08:45.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.762 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.762 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2023-12-12T10:08:45.762 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2023-12-12T10:08:45.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.763 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2023-12-12T10:08:45.763 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.763 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.763 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.764 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-12T10:08:45.764 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.764 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.764 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2023-12-12T10:08:45.764 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2023-12-12T10:08:45.764 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.765 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.765 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:45.765 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:45.765 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2023-12-12T10:08:45.765 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.765 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-12T10:08:45.766 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.766 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2023-12-12T10:08:45.766 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2023-12-12T10:08:45.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.766 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.767 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.767 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.767 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.767 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.767 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2023-12-12T10:08:45.767 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2023-12-12T10:08:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2023-12-12T10:08:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.769 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2023-12-12T10:08:45.769 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.769 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.769 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.769 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-12T10:08:45.769 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.769 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.770 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.770 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2023-12-12T10:08:45.770 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.770 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.771 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.771 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.771 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.771 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.771 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.771 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.772 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.772 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.772 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.772 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.772 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.772 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.772 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.773 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.773 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.773 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.773 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.773 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.774 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.774 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.774 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.774 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.774 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.775 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.775 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.776 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.776 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.776 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.776 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.777 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.777 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.777 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.777 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.777 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.777 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.777 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.778 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.778 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.778 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.779 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.779 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.779 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.780 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.780 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.780 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.781 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.781 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.781 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2023-12-12T10:08:45.781 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.781 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.781 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.782 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.782 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.782 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.782 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.783 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.783 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.783 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.783 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.783 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.783 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.784 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.784 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.784 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.784 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.784 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.784 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.785 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.785 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.785 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.786 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.786 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.786 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.787 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.787 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.787 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.787 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.787 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.788 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.788 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.788 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.788 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.788 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.788 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.788 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.788 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.789 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.789 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.789 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.789 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.790 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.790 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.790 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.790 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.790 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.790 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.791 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.791 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.791 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.791 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.791 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2023-12-12T10:08:45.792 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.792 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.792 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.792 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2023-12-12T10:08:45.792 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2023-12-12T10:08:45.792 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.792 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.793 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2023-12-12T10:08:45.793 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.793 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.793 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.793 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-12T10:08:45.793 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.794 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2023-12-12T10:08:45.794 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2023-12-12T10:08:45.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.794 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.795 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.795 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2023-12-12T10:08:45.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.795 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.795 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-12T10:08:45.795 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.795 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.796 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.796 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.796 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.796 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.797 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.797 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.797 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.797 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.797 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.798 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.798 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.798 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.798 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.798 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.798 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.799 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.799 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.799 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.799 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.799 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.800 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.800 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.800 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.800 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.801 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.801 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.801 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.802 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.802 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.802 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.802 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.802 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.802 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.802 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.803 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.803 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.803 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.803 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.804 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.804 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.804 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.804 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.804 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.805 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2023-12-12T10:08:45.805 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.805 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.805 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.805 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2023-12-12T10:08:45.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2023-12-12T10:08:45.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.806 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.806 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2023-12-12T10:08:45.806 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.806 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.806 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.806 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2023-12-12T10:08:45.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-12T10:08:45.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.807 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.807 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.807 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.808 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.808 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.808 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.808 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.809 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.809 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.809 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.809 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.809 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.809 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.809 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.810 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.810 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.810 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.810 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.811 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.811 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.811 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.812 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.812 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.812 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.812 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.813 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.813 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.813 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.813 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.813 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.813 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.814 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.814 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.814 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.815 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.815 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.816 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.816 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.816 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.816 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.816 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.816 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.816 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2023-12-12T10:08:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2023-12-12T10:08:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.818 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.818 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2023-12-12T10:08:45.818 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-12T10:08:45.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.818 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.819 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2023-12-12T10:08:45.819 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2023-12-12T10:08:45.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.819 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.819 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.819 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2023-12-12T10:08:45.819 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.819 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.820 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.820 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2023-12-12T10:08:45.820 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.820 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.820 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2023-12-12T10:08:45.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2023-12-12T10:08:45.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.821 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.821 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2023-12-12T10:08:45.821 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.821 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.821 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2023-12-12T10:08:45.822 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.822 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.822 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2023-12-12T10:08:45.822 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2023-12-12T10:08:45.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.823 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.823 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.823 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.823 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.823 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2023-12-12T10:08:45.823 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.824 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2023-12-12T10:08:45.824 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2023-12-12T10:08:45.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.824 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2023-12-12T10:08:45.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.825 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.825 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-12T10:08:45.825 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-12T10:08:45.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.825 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.826 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.826 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.826 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2023-12-12T10:08:45.826 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.826 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.826 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.827 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.827 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.828 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.828 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.828 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.828 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.828 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.828 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.828 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.829 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.829 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.831 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.831 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.831 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.832 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.833 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.833 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.833 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.833 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.833 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.833 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.834 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.835 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.835 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.836 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.836 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2023-12-12T10:08:45.837 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.837 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.837 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.837 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.838 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.838 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.838 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.839 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.839 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.839 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.839 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.839 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.839 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.839 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.840 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.840 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.840 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.840 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.841 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.841 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.841 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.841 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.841 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.841 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.842 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.842 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.842 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.842 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.842 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.843 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.843 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.843 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.843 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.843 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.843 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.844 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.844 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.844 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.844 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.844 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.845 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.845 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.845 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.845 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.845 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.845 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.846 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2023-12-12T10:08:45.847 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.847 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.847 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.847 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.848 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.848 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.848 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.848 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.848 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.848 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.849 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.849 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.849 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.849 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.849 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.849 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.850 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.850 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.850 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.850 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.851 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.851 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.851 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.852 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.852 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.852 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.852 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.852 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.853 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.853 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.853 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.853 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.853 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.853 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.853 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.854 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.854 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.854 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.855 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.856 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.856 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.856 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.857 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2023-12-12T10:08:45.857 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.857 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.857 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.858 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2023-12-12T10:08:45.858 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2023-12-12T10:08:45.858 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.858 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.859 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.859 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.859 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-12T10:08:45.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.859 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.859 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.860 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.860 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.860 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.861 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.861 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.861 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.861 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.861 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.861 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.861 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.862 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.862 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.862 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.862 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.862 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.863 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.863 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.863 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.863 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.864 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.864 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.864 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.864 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.864 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.865 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.865 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.865 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.865 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.865 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.865 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.865 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.866 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.866 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.866 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.866 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.866 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.866 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.866 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.867 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.867 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.867 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.867 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.867 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.868 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.868 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.868 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.868 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.868 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.868 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.869 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2023-12-12T10:08:45.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2023-12-12T10:08:45.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.869 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.869 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.869 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.869 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.870 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.870 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2023-12-12T10:08:45.870 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2023-12-12T10:08:45.870 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2023-12-12T10:08:45.870 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2023-12-12T10:08:45.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.871 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.871 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.871 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.871 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2023-12-12T10:08:45.871 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.872 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.872 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.872 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.872 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.872 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.873 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.873 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.873 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.873 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.873 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.874 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.874 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.874 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.874 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.874 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.874 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.875 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.875 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.875 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.875 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.875 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.875 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.876 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.876 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.876 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.876 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.877 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.877 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.877 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.877 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.877 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.878 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.878 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.878 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.878 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.878 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.878 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.878 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.878 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.879 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.879 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.879 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.879 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.880 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.880 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.880 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.880 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.880 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.880 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2023-12-12T10:08:45.881 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.882 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.883 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.884 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.885 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.885 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.885 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.885 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.885 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.886 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.886 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.886 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.886 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.886 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.886 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.887 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.887 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.887 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.887 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.887 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.887 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.887 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.888 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.888 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.888 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.888 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.888 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.888 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.888 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.888 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.889 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.889 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.890 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.891 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2023-12-12T10:08:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.893 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.894 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.896 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.897 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.898 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.899 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.900 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:45.901 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.902 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2023-12-12T10:08:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2023-12-12T10:08:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2023-12-12T10:08:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2023-12-12T10:08:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2023-12-12T10:08:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-12T10:08:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.904 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2023-12-12T10:08:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.905 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2023-12-12T10:08:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2023-12-12T10:08:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-12T10:08:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.907 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2023-12-12T10:08:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2023-12-12T10:08:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.910 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.911 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.913 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.914 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.915 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.916 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.917 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.918 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2023-12-12T10:08:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2023-12-12T10:08:45.919 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2023-12-12T10:08:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.920 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2023-12-12T10:08:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:45.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2023-12-12T10:08:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.923 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2023-12-12T10:08:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2023-12-12T10:08:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2023-12-12T10:08:45.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.925 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2023-12-12T10:08:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2023-12-12T10:08:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.926 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-12T10:08:45.927 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-12T10:08:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.928 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.929 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.933 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.934 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.935 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.936 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.937 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2023-12-12T10:08:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2023-12-12T10:08:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.938 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-12T10:08:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-12T10:08:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2023-12-12T10:08:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2023-12-12T10:08:45.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2023-12-12T10:08:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2023-12-12T10:08:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.942 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2023-12-12T10:08:45.943 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2023-12-12T10:08:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2023-12-12T10:08:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.944 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-12T10:08:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2023-12-12T10:08:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2023-12-12T10:08:45.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2023-12-12T10:08:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-12T10:08:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2023-12-12T10:08:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2023-12-12T10:08:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.947 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2023-12-12T10:08:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2023-12-12T10:08:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2023-12-12T10:08:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.949 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2023-12-12T10:08:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2023-12-12T10:08:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2023-12-12T10:08:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.950 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2023-12-12T10:08:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2023-12-12T10:08:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2023-12-12T10:08:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2023-12-12T10:08:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2023-12-12T10:08:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2023-12-12T10:08:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.953 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.955 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.956 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.957 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.958 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.959 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.960 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.961 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.962 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2023-12-12T10:08:45.963 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2023-12-12T10:08:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.964 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2023-12-12T10:08:45.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2023-12-12T10:08:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.966 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2023-12-12T10:08:45.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2023-12-12T10:08:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.968 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2023-12-12T10:08:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2023-12-12T10:08:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2023-12-12T10:08:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.970 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2023-12-12T10:08:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2023-12-12T10:08:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.972 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.974 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.975 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.976 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.977 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.979 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.980 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2023-12-12T10:08:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2023-12-12T10:08:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.984 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2023-12-12T10:08:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2023-12-12T10:08:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.986 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2023-12-12T10:08:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.988 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.989 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.990 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:45.993 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.994 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:45.995 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.996 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2023-12-12T10:08:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:45.997 INFO:teuthology.orchestra.run.smithi042.stdout: "drive_group_interval": { 2023-12-12T10:08:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "drive_group_interval", 2023-12-12T10:08:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300.0", 2023-12-12T10:08:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:45.998 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-12T10:08:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:45.999 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.000 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.001 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.001 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.001 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.001 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.001 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.001 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.002 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.002 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.002 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.002 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.002 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.003 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.003 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.003 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.004 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.004 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.004 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.004 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.004 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.004 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.005 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.005 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.005 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.006 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.006 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.006 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.006 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.006 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.006 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.006 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.007 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.007 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.007 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.008 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.008 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.008 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.008 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.009 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.009 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.009 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.009 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.009 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2023-12-12T10:08:46.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2023-12-12T10:08:46.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.010 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.010 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2023-12-12T10:08:46.010 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.010 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.011 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-12T10:08:46.011 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.011 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.011 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2023-12-12T10:08:46.012 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.012 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.012 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.012 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.013 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.013 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.013 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.013 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.014 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.014 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.014 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.014 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.014 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.014 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.014 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.015 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.015 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.015 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.015 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.015 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.015 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.015 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.016 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.016 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.016 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.016 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.016 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.017 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.017 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.017 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.017 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.017 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.017 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.018 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.018 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.018 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.018 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.018 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.018 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.018 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.018 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.019 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.019 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.019 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.019 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.019 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.019 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.019 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.020 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.020 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.020 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.020 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.020 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.021 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.021 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.021 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.021 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.021 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.021 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.021 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.022 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.022 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.022 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2023-12-12T10:08:46.022 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2023-12-12T10:08:46.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.022 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.023 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.023 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.023 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.023 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.023 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.023 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.024 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.024 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2023-12-12T10:08:46.024 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2023-12-12T10:08:46.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.024 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.025 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2023-12-12T10:08:46.025 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.025 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.025 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.025 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.025 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.026 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.026 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.026 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2023-12-12T10:08:46.026 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2023-12-12T10:08:46.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.026 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.026 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.027 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.027 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.027 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.027 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.027 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.027 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.027 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.028 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2023-12-12T10:08:46.028 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2023-12-12T10:08:46.028 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.028 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.028 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.028 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.028 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.029 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.029 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.029 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.029 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.029 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.029 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.030 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2023-12-12T10:08:46.030 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2023-12-12T10:08:46.030 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:46.030 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.030 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.030 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.031 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.031 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.031 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.031 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.031 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.032 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2023-12-12T10:08:46.032 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2023-12-12T10:08:46.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.032 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.032 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.032 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.033 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.033 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.033 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.033 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.033 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.033 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.033 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.033 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2023-12-12T10:08:46.034 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2023-12-12T10:08:46.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.034 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.034 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.034 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.034 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.034 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.035 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.035 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.035 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.035 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.035 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2023-12-12T10:08:46.035 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2023-12-12T10:08:46.036 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.036 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.036 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.036 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.036 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.036 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.036 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.036 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.037 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.037 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.037 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.037 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2023-12-12T10:08:46.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2023-12-12T10:08:46.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.037 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2023-12-12T10:08:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.038 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2023-12-12T10:08:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2023-12-12T10:08:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.040 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.040 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.040 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.040 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.040 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.040 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.041 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2023-12-12T10:08:46.041 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2023-12-12T10:08:46.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.041 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.041 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.041 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.042 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.042 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.042 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.042 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.042 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.042 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.043 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2023-12-12T10:08:46.043 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2023-12-12T10:08:46.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.043 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.043 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.043 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.044 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.044 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.044 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.044 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.044 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.044 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.045 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.045 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2023-12-12T10:08:46.045 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.045 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.045 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.045 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2023-12-12T10:08:46.046 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2023-12-12T10:08:46.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.046 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.046 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.046 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.046 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.046 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.047 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2023-12-12T10:08:46.047 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.047 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.047 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2023-12-12T10:08:46.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2023-12-12T10:08:46.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.048 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.048 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.048 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.048 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.048 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.048 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2023-12-12T10:08:46.048 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.049 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.049 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.049 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.049 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.049 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.050 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.050 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.050 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.050 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.050 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.050 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.050 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.050 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.051 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.051 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.051 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.051 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.051 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.051 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.052 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.052 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.052 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.052 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.052 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.052 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.053 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.053 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.053 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.053 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.053 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.053 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.054 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.054 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.054 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.054 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.054 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.055 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.055 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.055 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.055 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.055 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.055 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.055 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.056 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.056 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.056 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.056 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.056 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.056 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.057 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.057 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.057 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.057 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.057 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.057 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.057 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.057 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.058 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.058 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.058 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2023-12-12T10:08:46.058 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.058 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.059 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.059 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.059 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.059 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.059 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.060 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.060 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.060 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.060 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.060 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.060 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.060 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.060 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.061 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.061 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.061 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.061 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.061 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.061 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.062 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.062 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.062 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.062 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.062 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.063 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.063 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.063 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.063 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.063 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.063 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.063 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.064 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.064 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.064 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.064 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.064 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.064 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.064 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.065 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.065 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.065 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.065 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.065 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.065 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.065 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.065 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.066 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.066 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.066 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.066 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.067 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.067 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.067 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.067 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.067 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.067 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.067 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.068 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.068 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.068 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.068 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2023-12-12T10:08:46.068 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.069 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.069 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.069 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.069 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.069 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.069 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.070 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.070 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.070 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.070 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.070 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.070 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.070 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.071 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.071 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.071 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.071 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.071 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.071 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.072 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.072 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.072 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.072 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.072 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.072 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.072 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.073 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.073 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.073 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.073 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.073 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.073 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.073 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.074 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.074 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.074 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.074 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.074 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.074 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.074 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.075 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.075 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.075 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.075 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.075 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.075 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.075 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.075 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.076 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.076 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.076 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.076 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.077 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.077 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.077 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.077 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.077 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.077 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.077 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.078 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.078 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.078 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2023-12-12T10:08:46.078 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.078 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.078 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.079 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2023-12-12T10:08:46.079 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2023-12-12T10:08:46.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.079 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.079 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-12T10:08:46.079 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.080 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.080 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.080 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.080 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.080 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:46.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:46.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.081 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.081 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2023-12-12T10:08:46.081 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.081 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.081 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.082 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.082 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.082 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.083 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.083 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.083 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.083 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.083 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.083 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.083 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.083 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.084 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.084 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.084 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.084 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.084 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.084 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.085 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.085 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.085 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.085 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.086 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.086 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.086 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.086 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.086 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.087 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.087 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.087 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.087 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.087 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.088 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.088 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.088 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.088 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.088 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.088 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.088 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.089 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.089 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.089 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.089 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.089 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.089 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.090 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.090 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.090 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.090 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.090 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.090 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.090 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.091 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.091 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.091 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.091 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.091 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.092 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2023-12-12T10:08:46.092 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.092 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.092 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.092 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2023-12-12T10:08:46.092 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2023-12-12T10:08:46.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.092 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.093 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.093 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.093 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.093 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.093 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-12T10:08:46.093 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.094 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.094 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2023-12-12T10:08:46.094 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2023-12-12T10:08:46.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.094 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.095 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.095 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.095 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.095 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.095 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-12T10:08:46.095 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.095 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.096 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.096 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2023-12-12T10:08:46.096 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2023-12-12T10:08:46.096 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.096 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.096 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.096 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.097 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.097 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.097 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.097 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-12T10:08:46.097 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.097 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.098 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.098 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2023-12-12T10:08:46.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2023-12-12T10:08:46.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.098 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.098 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.099 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.099 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.099 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-12T10:08:46.099 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.099 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2023-12-12T10:08:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2023-12-12T10:08:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.100 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-12T10:08:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.101 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2023-12-12T10:08:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2023-12-12T10:08:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.102 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.103 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.103 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.103 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.103 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2023-12-12T10:08:46.103 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2023-12-12T10:08:46.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.104 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.105 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.105 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2023-12-12T10:08:46.105 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2023-12-12T10:08:46.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.105 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.106 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-12T10:08:46.106 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.106 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.106 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.106 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.106 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.106 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.107 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2023-12-12T10:08:46.107 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2023-12-12T10:08:46.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.107 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.107 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.107 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.108 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.108 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.108 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.108 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.108 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.108 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:46.109 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:46.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.109 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.109 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2023-12-12T10:08:46.109 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2023-12-12T10:08:46.109 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.109 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.110 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.110 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.110 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.110 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2023-12-12T10:08:46.110 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2023-12-12T10:08:46.110 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.111 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.111 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2023-12-12T10:08:46.111 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.111 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.111 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.111 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.111 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.112 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.112 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.112 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2023-12-12T10:08:46.112 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2023-12-12T10:08:46.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.112 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.113 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.113 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.113 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.113 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.113 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.113 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.113 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.114 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.114 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2023-12-12T10:08:46.114 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2023-12-12T10:08:46.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.114 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.114 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.114 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.115 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.115 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.115 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.115 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.115 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.115 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.115 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.116 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.116 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.116 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.116 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.116 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.117 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.117 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.117 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.117 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.117 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.117 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.117 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.118 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.118 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.118 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.118 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.118 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.119 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.119 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.119 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.119 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.119 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.119 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.120 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.120 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.120 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.120 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.121 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.121 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.121 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.121 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.121 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.121 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.121 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.122 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.123 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.124 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2023-12-12T10:08:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2023-12-12T10:08:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.125 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.126 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.126 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.126 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.126 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.126 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.126 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.127 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2023-12-12T10:08:46.127 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2023-12-12T10:08:46.127 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.127 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.127 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.127 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.127 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.127 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.128 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.128 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.128 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.128 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.128 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2023-12-12T10:08:46.128 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2023-12-12T10:08:46.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.129 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.129 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-12T10:08:46.129 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.129 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.129 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.130 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.130 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.130 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.130 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.131 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2023-12-12T10:08:46.131 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.131 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.131 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.131 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.132 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.132 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.132 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.132 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.132 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.133 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.133 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.133 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.133 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.133 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.133 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.133 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.133 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.134 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.134 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.134 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.134 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.134 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.135 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.135 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.135 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.135 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.135 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.135 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.135 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.136 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.136 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.136 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.136 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.136 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.137 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.137 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.137 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.137 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.137 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.137 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.137 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.137 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.138 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.138 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.138 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.138 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.138 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.138 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.139 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.139 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.139 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.139 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.139 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.139 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.139 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.140 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.140 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.140 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.140 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.141 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.141 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2023-12-12T10:08:46.141 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.141 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.141 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.141 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.141 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.141 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.142 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.142 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.142 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.142 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.142 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.142 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.143 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.143 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.143 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.143 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.143 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.143 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.143 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.144 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.144 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.144 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.144 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.144 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.145 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.145 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.145 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.145 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.145 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.145 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.146 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.146 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.146 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.146 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.147 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.147 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.147 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.147 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.147 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.147 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.147 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.148 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.148 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.148 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.148 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.148 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.148 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.148 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.149 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.149 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.149 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.149 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.149 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.149 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.150 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.150 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.150 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.150 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.150 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.150 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.151 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.151 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2023-12-12T10:08:46.151 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2023-12-12T10:08:46.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.151 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.152 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2023-12-12T10:08:46.152 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.152 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.152 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.152 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-12T10:08:46.152 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2023-12-12T10:08:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2023-12-12T10:08:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2023-12-12T10:08:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2023-12-12T10:08:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2023-12-12T10:08:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.155 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-12T10:08:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.156 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2023-12-12T10:08:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.157 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2023-12-12T10:08:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2023-12-12T10:08:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2023-12-12T10:08:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.158 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2023-12-12T10:08:46.159 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2023-12-12T10:08:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.160 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.161 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.164 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.166 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.167 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.168 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.169 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.170 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.171 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2023-12-12T10:08:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2023-12-12T10:08:46.172 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.173 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2023-12-12T10:08:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2023-12-12T10:08:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.174 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2023-12-12T10:08:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2023-12-12T10:08:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2023-12-12T10:08:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.176 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-12T10:08:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2023-12-12T10:08:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2023-12-12T10:08:46.178 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard", 2023-12-12T10:08:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2023-12-12T10:08:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2023-12-12T10:08:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "restful", 2023-12-12T10:08:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "snap_schedule" 2023-12-12T10:08:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2023-12-12T10:08:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.179 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2023-12-12T10:08:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2023-12-12T10:08:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.181 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.183 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.184 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.185 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.186 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.187 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.188 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.189 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.190 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2023-12-12T10:08:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2023-12-12T10:08:46.191 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.192 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2023-12-12T10:08:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2023-12-12T10:08:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2023-12-12T10:08:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.193 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2023-12-12T10:08:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2023-12-12T10:08:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.194 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2023-12-12T10:08:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2023-12-12T10:08:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.195 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2023-12-12T10:08:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2023-12-12T10:08:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2023-12-12T10:08:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.197 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2023-12-12T10:08:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2023-12-12T10:08:46.198 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2023-12-12T10:08:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2023-12-12T10:08:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.199 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2023-12-12T10:08:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2023-12-12T10:08:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2023-12-12T10:08:46.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2023-12-12T10:08:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.201 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2023-12-12T10:08:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2023-12-12T10:08:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-12T10:08:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.203 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2023-12-12T10:08:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2023-12-12T10:08:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.204 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2023-12-12T10:08:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.205 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2023-12-12T10:08:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2023-12-12T10:08:46.206 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2023-12-12T10:08:46.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.207 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.207 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.207 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.207 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.207 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.207 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2023-12-12T10:08:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2023-12-12T10:08:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2023-12-12T10:08:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.208 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2023-12-12T10:08:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2023-12-12T10:08:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-12T10:08:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.209 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2023-12-12T10:08:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2023-12-12T10:08:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.210 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2023-12-12T10:08:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2023-12-12T10:08:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-12T10:08:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-12T10:08:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.211 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2023-12-12T10:08:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2023-12-12T10:08:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2023-12-12T10:08:46.212 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2023-12-12T10:08:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-12T10:08:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2023-12-12T10:08:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2023-12-12T10:08:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2023-12-12T10:08:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.214 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-12T10:08:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-12T10:08:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.215 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2023-12-12T10:08:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2023-12-12T10:08:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2023-12-12T10:08:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2023-12-12T10:08:46.216 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2023-12-12T10:08:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2023-12-12T10:08:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-12T10:08:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.217 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2023-12-12T10:08:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2023-12-12T10:08:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2023-12-12T10:08:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.218 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2023-12-12T10:08:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-12T10:08:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.219 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2023-12-12T10:08:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2023-12-12T10:08:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.220 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2023-12-12T10:08:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2023-12-12T10:08:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-12T10:08:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-12T10:08:46.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.223 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.224 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.226 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.227 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.229 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.230 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.231 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.233 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2023-12-12T10:08:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2023-12-12T10:08:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.234 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-12T10:08:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.235 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2023-12-12T10:08:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2023-12-12T10:08:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2023-12-12T10:08:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2023-12-12T10:08:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2023-12-12T10:08:46.237 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap" 2023-12-12T10:08:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2023-12-12T10:08:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.238 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2023-12-12T10:08:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2023-12-12T10:08:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.239 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-12T10:08:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:46.240 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.241 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-12T10:08:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2023-12-12T10:08:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2023-12-12T10:08:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.242 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-12T10:08:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-12T10:08:46.243 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2023-12-12T10:08:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2023-12-12T10:08:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.244 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-12T10:08:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.245 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2023-12-12T10:08:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.246 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2023-12-12T10:08:46.247 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2023-12-12T10:08:46.247 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:46.247 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.247 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.247 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2023-12-12T10:08:46.247 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.247 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.247 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.248 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-12T10:08:46.248 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.248 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.248 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2023-12-12T10:08:46.248 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2023-12-12T10:08:46.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.249 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.249 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.249 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2023-12-12T10:08:46.249 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.249 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.249 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.249 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-12T10:08:46.249 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.250 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.250 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2023-12-12T10:08:46.250 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2023-12-12T10:08:46.250 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.250 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2023-12-12T10:08:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-12T10:08:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.251 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2023-12-12T10:08:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2023-12-12T10:08:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.252 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-12T10:08:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-12T10:08:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.253 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2023-12-12T10:08:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2023-12-12T10:08:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.254 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-12T10:08:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2023-12-12T10:08:46.255 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2023-12-12T10:08:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2023-12-12T10:08:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.256 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-12T10:08:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2023-12-12T10:08:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2023-12-12T10:08:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.257 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2023-12-12T10:08:46.258 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2023-12-12T10:08:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2023-12-12T10:08:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2023-12-12T10:08:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.259 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2023-12-12T10:08:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2023-12-12T10:08:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.260 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-12T10:08:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2023-12-12T10:08:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2023-12-12T10:08:46.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-12T10:08:46.262 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2023-12-12T10:08:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2023-12-12T10:08:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-12T10:08:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.264 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2023-12-12T10:08:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2023-12-12T10:08:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.265 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-12T10:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2023-12-12T10:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2023-12-12T10:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-12T10:08:46.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2023-12-12T10:08:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2023-12-12T10:08:46.268 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2023-12-12T10:08:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-12T10:08:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.269 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2023-12-12T10:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2023-12-12T10:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2023-12-12T10:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.270 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-12T10:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2023-12-12T10:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2023-12-12T10:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.272 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-12T10:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2023-12-12T10:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2023-12-12T10:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2023-12-12T10:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.274 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-12T10:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2023-12-12T10:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.275 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2023-12-12T10:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2023-12-12T10:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.276 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-12T10:08:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2023-12-12T10:08:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.277 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2023-12-12T10:08:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2023-12-12T10:08:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-12T10:08:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2023-12-12T10:08:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.279 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2023-12-12T10:08:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2023-12-12T10:08:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.280 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-12T10:08:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2023-12-12T10:08:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2023-12-12T10:08:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2023-12-12T10:08:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-12T10:08:46.282 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2023-12-12T10:08:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.283 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2023-12-12T10:08:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2023-12-12T10:08:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-12T10:08:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.284 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2023-12-12T10:08:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2023-12-12T10:08:46.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:latest", 2023-12-12T10:08:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2023-12-12T10:08:46.286 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2023-12-12T10:08:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2023-12-12T10:08:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-12T10:08:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2023-12-12T10:08:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.288 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2023-12-12T10:08:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2023-12-12T10:08:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-12T10:08:46.289 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2023-12-12T10:08:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.290 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2023-12-12T10:08:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2023-12-12T10:08:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-12T10:08:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.291 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2023-12-12T10:08:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2023-12-12T10:08:46.292 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2023-12-12T10:08:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.293 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2023-12-12T10:08:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2023-12-12T10:08:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2023-12-12T10:08:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.294 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-12T10:08:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2023-12-12T10:08:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2023-12-12T10:08:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2023-12-12T10:08:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-12T10:08:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.297 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2023-12-12T10:08:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2023-12-12T10:08:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2023-12-12T10:08:46.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-12T10:08:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.299 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2023-12-12T10:08:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2023-12-12T10:08:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2023-12-12T10:08:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.300 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2023-12-12T10:08:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2023-12-12T10:08:46.301 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2023-12-12T10:08:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.302 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-12T10:08:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2023-12-12T10:08:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2023-12-12T10:08:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.303 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2023-12-12T10:08:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.304 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2023-12-12T10:08:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2023-12-12T10:08:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.305 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2023-12-12T10:08:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2023-12-12T10:08:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2023-12-12T10:08:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.307 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2023-12-12T10:08:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.308 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.309 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.310 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.311 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2023-12-12T10:08:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2023-12-12T10:08:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.312 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-12T10:08:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.313 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.314 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-12T10:08:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.316 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.317 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.319 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-12T10:08:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-12T10:08:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.320 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-12T10:08:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.321 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-12T10:08:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-12T10:08:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2023-12-12T10:08:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.322 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-12T10:08:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2023-12-12T10:08:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2023-12-12T10:08:46.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.324 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2023-12-12T10:08:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2023-12-12T10:08:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2023-12-12T10:08:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.325 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-12T10:08:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.326 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2023-12-12T10:08:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2023-12-12T10:08:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2023-12-12T10:08:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.328 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2023-12-12T10:08:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2023-12-12T10:08:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2023-12-12T10:08:46.329 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2023-12-12T10:08:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2023-12-12T10:08:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2023-12-12T10:08:46.330 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2023-12-12T10:08:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2023-12-12T10:08:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.331 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-12T10:08:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-12T10:08:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.332 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2023-12-12T10:08:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2023-12-12T10:08:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-12T10:08:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.334 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2023-12-12T10:08:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2023-12-12T10:08:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.335 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-12T10:08:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2023-12-12T10:08:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2023-12-12T10:08:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.337 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-12T10:08:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.338 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2023-12-12T10:08:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2023-12-12T10:08:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.339 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-12T10:08:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2023-12-12T10:08:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2023-12-12T10:08:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.341 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-12T10:08:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.342 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2023-12-12T10:08:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2023-12-12T10:08:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.343 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2023-12-12T10:08:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2023-12-12T10:08:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.344 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2023-12-12T10:08:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2023-12-12T10:08:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.345 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-12T10:08:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2023-12-12T10:08:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2023-12-12T10:08:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.347 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-12T10:08:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.348 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2023-12-12T10:08:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2023-12-12T10:08:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.349 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-12T10:08:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2023-12-12T10:08:46.350 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2023-12-12T10:08:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.351 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2023-12-12T10:08:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2023-12-12T10:08:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2023-12-12T10:08:46.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-12T10:08:46.353 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2023-12-12T10:08:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2023-12-12T10:08:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.354 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-12T10:08:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2023-12-12T10:08:46.356 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.358 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.360 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.361 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.362 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.363 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.364 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.365 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2023-12-12T10:08:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2023-12-12T10:08:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2023-12-12T10:08:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-12T10:08:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.367 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2023-12-12T10:08:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2023-12-12T10:08:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2023-12-12T10:08:46.368 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-12T10:08:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.369 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2023-12-12T10:08:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.370 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-12T10:08:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-12T10:08:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.371 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2023-12-12T10:08:46.372 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-12T10:08:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.373 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-12T10:08:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-12T10:08:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.374 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.375 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2023-12-12T10:08:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2023-12-12T10:08:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.376 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.377 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-12T10:08:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-12T10:08:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.378 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2023-12-12T10:08:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-12T10:08:46.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.380 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-12T10:08:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-12T10:08:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.381 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.382 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-12T10:08:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-12T10:08:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.383 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-12T10:08:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-12T10:08:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.385 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-12T10:08:46.386 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-12T10:08:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.387 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-12T10:08:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-12T10:08:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.388 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-12T10:08:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-12T10:08:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.390 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-12T10:08:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-12T10:08:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.393 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-12T10:08:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-12T10:08:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.394 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2023-12-12T10:08:46.395 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-12T10:08:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2023-12-12T10:08:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.396 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-12T10:08:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-12T10:08:46.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.398 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2023-12-12T10:08:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2023-12-12T10:08:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.400 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2023-12-12T10:08:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-12T10:08:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2023-12-12T10:08:46.401 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.402 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-12T10:08:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-12T10:08:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-12T10:08:46.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-12T10:08:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.405 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-12T10:08:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-12T10:08:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.406 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.407 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-12T10:08:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-12T10:08:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2023-12-12T10:08:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-12T10:08:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.410 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.411 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-12T10:08:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-12T10:08:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-12T10:08:46.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-12T10:08:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.414 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-12T10:08:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-12T10:08:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.415 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.416 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-12T10:08:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-12T10:08:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-12T10:08:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-12T10:08:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-12T10:08:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-12T10:08:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.421 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-12T10:08:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-12T10:08:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.423 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-12T10:08:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-12T10:08:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.425 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2023-12-12T10:08:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-12T10:08:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.427 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-12T10:08:46.428 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-12T10:08:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-12T10:08:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.429 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-12T10:08:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-12T10:08:46.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.431 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-12T10:08:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-12T10:08:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.432 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2023-12-12T10:08:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.433 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-12T10:08:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-12T10:08:46.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2023-12-12T10:08:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.435 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2023-12-12T10:08:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-12T10:08:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.437 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-12T10:08:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-12T10:08:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.438 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2023-12-12T10:08:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2023-12-12T10:08:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-12T10:08:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.440 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.441 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2023-12-12T10:08:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-12T10:08:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.442 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "UNSAFE_TLS_v1_2": { 2023-12-12T10:08:46.443 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "UNSAFE_TLS_v1_2", 2023-12-12T10:08:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.444 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-12T10:08:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-12T10:08:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.445 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.446 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-12T10:08:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-12T10:08:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.447 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.448 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-12T10:08:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-12T10:08:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:46.449 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.450 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2023-12-12T10:08:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2023-12-12T10:08:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.451 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2023-12-12T10:08:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2023-12-12T10:08:46.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.453 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2023-12-12T10:08:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2023-12-12T10:08:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.454 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2023-12-12T10:08:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2023-12-12T10:08:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2023-12-12T10:08:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2023-12-12T10:08:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.457 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2023-12-12T10:08:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2023-12-12T10:08:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.458 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.460 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.461 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.463 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.465 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.467 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2023-12-12T10:08:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2023-12-12T10:08:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.469 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2023-12-12T10:08:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2023-12-12T10:08:46.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2023-12-12T10:08:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2023-12-12T10:08:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2023-12-12T10:08:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.472 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2023-12-12T10:08:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.473 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2023-12-12T10:08:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2023-12-12T10:08:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.474 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2023-12-12T10:08:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2023-12-12T10:08:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2023-12-12T10:08:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.476 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2023-12-12T10:08:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2023-12-12T10:08:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2023-12-12T10:08:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.478 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2023-12-12T10:08:46.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2023-12-12T10:08:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2023-12-12T10:08:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.480 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.481 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2023-12-12T10:08:46.481 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.481 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.481 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.481 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.481 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2023-12-12T10:08:46.482 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2023-12-12T10:08:46.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.482 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2023-12-12T10:08:46.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2023-12-12T10:08:46.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2023-12-12T10:08:46.483 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.483 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.483 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.483 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.483 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2023-12-12T10:08:46.483 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2023-12-12T10:08:46.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.484 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.484 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.484 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.484 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.484 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.485 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.485 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.485 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.485 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.485 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2023-12-12T10:08:46.486 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.486 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.486 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.486 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2023-12-12T10:08:46.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2023-12-12T10:08:46.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.487 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.487 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.487 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.487 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.487 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.487 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.487 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2023-12-12T10:08:46.487 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.488 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.488 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.488 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.489 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.489 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.489 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.489 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.489 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.489 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.489 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.489 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.490 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.490 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.490 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.490 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.490 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.490 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.491 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.491 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.491 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.491 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.492 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.492 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.492 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.492 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.493 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.493 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.493 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.493 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.493 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.494 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.494 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.494 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.494 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.494 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.494 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.494 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.495 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.495 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.495 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.495 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.495 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.495 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.496 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.496 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.496 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.496 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.497 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.497 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.497 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.497 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2023-12-12T10:08:46.497 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2023-12-12T10:08:46.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.498 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.498 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.498 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2023-12-12T10:08:46.498 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.498 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.498 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-12T10:08:46.499 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.499 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.499 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2023-12-12T10:08:46.499 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2023-12-12T10:08:46.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.499 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.500 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2023-12-12T10:08:46.500 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.500 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.500 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.500 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-12T10:08:46.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.501 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.501 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2023-12-12T10:08:46.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2023-12-12T10:08:46.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.501 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.501 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2023-12-12T10:08:46.502 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.502 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.502 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.502 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2023-12-12T10:08:46.502 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.502 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.503 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2023-12-12T10:08:46.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2023-12-12T10:08:46.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.503 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.503 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2023-12-12T10:08:46.504 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.504 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.504 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.504 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-12T10:08:46.504 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.504 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.505 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2023-12-12T10:08:46.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2023-12-12T10:08:46.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.505 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.505 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.505 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.506 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.506 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.506 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-12T10:08:46.506 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.506 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.506 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:46.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:46.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.507 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:46.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.507 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.508 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.508 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2023-12-12T10:08:46.508 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.508 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.508 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2023-12-12T10:08:46.508 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2023-12-12T10:08:46.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.509 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.509 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2023-12-12T10:08:46.509 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.509 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.509 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.510 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-12T10:08:46.510 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.510 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.510 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.510 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.511 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.511 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2023-12-12T10:08:46.511 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.511 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.511 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.511 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.511 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.512 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.512 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.512 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.512 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.512 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.512 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.513 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.513 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.513 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.513 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.513 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.513 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.513 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.513 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.514 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.514 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.514 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.514 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.514 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.514 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.515 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.515 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.515 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.516 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.516 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.516 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.516 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.517 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.517 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.517 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.517 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.517 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.517 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.517 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.517 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.517 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.518 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.518 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.518 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.518 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.518 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.519 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.519 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.519 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.519 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.519 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.520 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.520 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.520 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.520 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2023-12-12T10:08:46.520 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2023-12-12T10:08:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2023-12-12T10:08:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.521 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.522 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.522 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.522 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2023-12-12T10:08:46.522 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2023-12-12T10:08:46.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.522 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.523 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2023-12-12T10:08:46.523 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.523 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.523 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.523 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.523 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.524 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.524 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:46.524 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:46.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.524 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.524 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2023-12-12T10:08:46.525 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.525 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.525 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.525 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.525 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.525 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2023-12-12T10:08:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2023-12-12T10:08:46.526 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2023-12-12T10:08:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2023-12-12T10:08:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2023-12-12T10:08:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2023-12-12T10:08:46.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.528 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.528 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.528 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-12T10:08:46.528 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.528 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.528 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.528 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.529 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2023-12-12T10:08:46.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2023-12-12T10:08:46.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.529 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2023-12-12T10:08:46.529 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.529 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.530 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-12T10:08:46.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.530 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.530 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2023-12-12T10:08:46.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2023-12-12T10:08:46.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.531 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.531 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.531 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.531 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.531 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.532 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2023-12-12T10:08:46.532 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.532 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.532 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:46.532 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:46.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.533 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.533 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2023-12-12T10:08:46.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2023-12-12T10:08:46.533 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.533 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.533 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-12T10:08:46.534 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.534 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.534 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.534 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.535 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.535 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.535 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.535 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.535 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.536 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.536 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.536 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.536 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.536 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.536 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.536 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.536 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.537 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.537 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.537 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.538 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.538 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.538 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.538 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.538 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.539 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.539 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.539 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.539 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.540 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.540 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.540 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.540 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.540 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.540 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.540 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.541 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.541 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.541 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.541 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.541 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.541 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.541 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.542 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.542 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.542 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.542 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.542 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.543 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.543 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.543 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.543 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.543 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.544 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2023-12-12T10:08:46.544 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2023-12-12T10:08:46.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.544 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.545 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.545 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.545 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.545 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2023-12-12T10:08:46.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.546 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2023-12-12T10:08:46.546 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2023-12-12T10:08:46.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.546 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.546 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2023-12-12T10:08:46.547 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.547 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.547 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.547 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2023-12-12T10:08:46.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.547 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.548 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2023-12-12T10:08:46.548 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2023-12-12T10:08:46.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.548 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.548 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2023-12-12T10:08:46.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.548 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.549 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.549 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-12T10:08:46.549 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.549 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2023-12-12T10:08:46.549 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2023-12-12T10:08:46.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.550 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.550 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:46.550 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:46.550 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2023-12-12T10:08:46.550 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.550 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-12T10:08:46.551 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.551 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.551 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2023-12-12T10:08:46.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2023-12-12T10:08:46.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.552 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.552 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.552 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.552 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2023-12-12T10:08:46.552 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.553 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.553 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2023-12-12T10:08:46.553 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2023-12-12T10:08:46.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.553 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.554 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2023-12-12T10:08:46.554 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.554 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.554 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.554 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-12T10:08:46.554 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.554 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.555 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.555 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2023-12-12T10:08:46.555 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.555 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.556 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.556 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.556 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.556 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.556 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.557 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.557 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.557 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.557 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.557 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.557 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.557 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.557 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.558 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.558 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.558 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.558 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.558 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.559 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.559 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.559 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.559 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.559 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.559 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.560 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.560 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.560 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.560 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.560 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.560 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.561 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.561 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.561 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.561 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.561 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.561 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.562 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.562 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.562 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.562 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.562 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.562 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.562 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.563 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.563 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.563 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.563 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.563 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.564 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.564 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.564 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.564 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.564 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.564 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.565 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.565 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.565 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.565 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2023-12-12T10:08:46.565 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.565 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.566 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.566 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.566 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.566 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.566 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.567 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.567 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.567 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.567 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.567 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.567 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.567 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.567 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.568 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.568 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.568 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.568 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.568 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.568 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.569 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.569 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.569 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.569 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.569 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.570 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.570 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.570 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.570 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.570 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.570 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.570 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.571 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.571 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.571 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.571 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.571 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.572 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.572 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.572 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.572 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.572 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.572 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.572 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.572 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.573 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.573 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.573 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.573 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.573 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.574 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.574 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.574 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.574 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.574 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.574 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.575 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.575 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.575 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.575 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2023-12-12T10:08:46.576 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.576 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.576 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.576 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2023-12-12T10:08:46.576 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2023-12-12T10:08:46.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.577 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.577 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.577 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2023-12-12T10:08:46.577 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.577 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.577 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.577 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-12T10:08:46.578 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.578 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.578 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.578 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2023-12-12T10:08:46.578 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2023-12-12T10:08:46.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.578 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.579 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:46.579 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2023-12-12T10:08:46.579 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.579 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.579 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-12T10:08:46.579 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.580 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.580 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.580 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.580 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.581 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.581 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.581 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.581 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.581 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.581 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.581 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.581 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.582 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.582 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.582 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.582 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.582 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.582 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.583 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.583 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.583 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.583 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.584 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.584 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.584 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.584 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.584 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.584 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.584 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.585 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.585 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.585 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.585 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.585 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.585 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.585 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.586 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.586 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.586 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.586 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.586 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.586 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.586 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.586 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.587 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.587 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.587 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.587 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.587 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.588 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.588 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.588 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.588 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.588 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.588 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.588 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.588 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.589 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.589 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.589 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2023-12-12T10:08:46.589 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.589 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.589 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.590 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2023-12-12T10:08:46.590 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2023-12-12T10:08:46.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.590 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.590 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2023-12-12T10:08:46.590 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.591 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.591 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.591 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2023-12-12T10:08:46.591 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-12T10:08:46.591 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.591 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.591 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.591 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.592 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.592 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.592 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.592 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.592 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.592 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.593 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.593 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.593 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.593 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.593 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.593 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.593 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.593 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.594 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.594 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.594 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.594 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.594 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.594 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.595 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.595 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.595 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.595 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.595 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.595 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.595 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.595 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.596 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.596 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.596 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.596 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.596 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.597 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.597 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.597 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.597 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.597 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.597 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.597 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.599 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.599 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.599 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.599 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.599 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.600 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.600 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.600 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.600 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.600 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2023-12-12T10:08:46.601 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2023-12-12T10:08:46.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.601 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.601 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.601 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.601 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.601 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.602 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2023-12-12T10:08:46.602 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-12T10:08:46.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.602 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.602 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2023-12-12T10:08:46.602 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2023-12-12T10:08:46.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.603 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.603 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2023-12-12T10:08:46.603 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.603 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.603 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.603 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2023-12-12T10:08:46.603 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.604 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.604 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.604 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2023-12-12T10:08:46.604 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2023-12-12T10:08:46.604 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.604 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.604 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.605 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2023-12-12T10:08:46.605 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.605 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.605 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.605 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2023-12-12T10:08:46.605 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.605 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.606 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2023-12-12T10:08:46.606 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2023-12-12T10:08:46.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.606 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.606 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.607 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.607 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.607 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.607 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2023-12-12T10:08:46.607 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.607 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.608 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2023-12-12T10:08:46.608 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2023-12-12T10:08:46.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.608 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.608 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2023-12-12T10:08:46.608 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.609 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.609 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-12T10:08:46.609 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-12T10:08:46.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.609 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.609 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.610 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.610 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2023-12-12T10:08:46.610 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.610 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.610 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.610 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.610 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.611 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.611 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.611 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.611 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.611 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.612 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.612 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.612 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.612 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.612 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.612 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.612 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.612 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.613 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.613 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.613 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.613 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.613 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.614 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.614 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.614 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.614 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.614 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.614 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.615 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.615 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.615 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.615 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.616 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.616 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.616 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.616 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.616 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.616 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.617 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.617 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.617 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.617 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.617 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.617 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.618 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.618 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.618 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.618 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.618 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.619 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.619 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.619 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.619 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2023-12-12T10:08:46.620 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.620 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.620 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.620 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.620 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.620 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.621 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.621 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.621 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.621 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.621 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.621 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.621 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.621 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.622 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.622 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.622 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.622 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.622 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.622 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.623 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.623 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.623 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.623 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.623 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.624 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.625 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.626 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.626 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.626 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.626 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.626 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.626 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.626 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.626 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.627 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.627 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.627 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.627 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.627 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.628 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.628 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.628 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.628 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.628 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.628 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.628 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.628 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.629 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.629 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.629 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2023-12-12T10:08:46.629 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.629 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.630 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.630 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.630 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.630 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.630 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.630 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.630 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.631 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.632 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.633 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.635 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.636 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2023-12-12T10:08:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.639 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2023-12-12T10:08:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2023-12-12T10:08:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-12T10:08:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.642 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.643 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.643 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.643 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.643 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.643 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.643 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.643 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.643 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.644 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.644 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.644 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.644 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.644 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.644 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.645 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.645 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.645 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.645 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.645 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.646 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.647 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.649 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2023-12-12T10:08:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2023-12-12T10:08:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2023-12-12T10:08:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2023-12-12T10:08:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2023-12-12T10:08:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2023-12-12T10:08:46.652 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.653 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.653 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.653 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.653 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2023-12-12T10:08:46.653 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.654 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.654 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.654 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.654 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.655 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.655 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.655 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.655 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.655 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.655 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.655 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.655 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.656 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.656 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.656 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.656 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.656 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.656 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.656 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.657 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.657 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.657 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.657 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.658 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.658 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.658 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.658 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.658 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.658 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.658 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.659 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.659 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.659 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.659 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.659 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.659 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.660 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.660 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.660 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.660 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.660 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.660 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.660 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.660 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.661 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.661 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.661 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.662 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.662 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.662 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.663 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.663 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.663 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2023-12-12T10:08:46.663 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.663 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.664 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.664 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.664 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.664 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.664 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.664 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.665 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.665 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.665 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.665 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.665 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.665 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.665 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.665 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.666 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.666 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.666 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.666 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.666 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.666 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.667 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.667 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.667 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.667 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.668 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.668 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.668 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.668 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.668 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.668 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.669 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.669 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.669 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.669 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.669 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.669 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.669 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.670 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.670 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.670 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.670 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.670 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.670 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.670 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.671 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.671 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.671 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.672 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.672 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.673 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.673 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.673 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.673 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2023-12-12T10:08:46.673 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.674 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.674 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.674 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.674 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.674 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.675 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.675 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.675 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.675 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.675 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.675 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.675 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.675 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.676 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.676 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.676 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.676 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.676 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.677 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.677 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.677 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.677 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.677 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.677 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.677 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.678 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.678 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.678 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.678 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.678 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.678 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.678 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.678 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.679 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.679 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.680 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.680 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.680 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.680 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.680 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.680 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.680 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.681 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.681 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.681 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.681 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.681 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.682 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.682 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.682 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.682 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.682 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.682 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.682 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.682 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.683 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:46.683 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:46.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.683 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:46.684 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.684 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.684 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.684 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.684 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.684 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.684 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.684 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.685 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2023-12-12T10:08:46.685 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2023-12-12T10:08:46.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:46.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.685 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2023-12-12T10:08:46.685 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2023-12-12T10:08:46.685 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2023-12-12T10:08:46.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-12T10:08:46.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.686 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.686 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.687 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2023-12-12T10:08:46.687 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.687 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.687 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.687 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2023-12-12T10:08:46.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2023-12-12T10:08:46.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.688 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.688 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.688 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.688 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.689 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-12T10:08:46.689 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.689 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.689 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.689 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2023-12-12T10:08:46.689 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2023-12-12T10:08:46.689 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.690 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.690 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.690 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.690 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.690 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.690 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.690 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.691 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.691 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.691 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.691 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.692 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.692 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.692 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.692 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.692 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.692 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.692 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.692 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.693 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.693 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.693 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.693 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.693 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.693 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.694 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.694 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.694 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.694 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.695 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.695 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.695 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.695 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.696 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.696 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.696 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.696 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.697 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.697 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.697 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.697 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.697 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.697 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.697 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.697 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.698 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.698 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.698 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.698 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.698 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.698 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.699 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.699 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.699 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.700 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.700 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2023-12-12T10:08:46.700 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2023-12-12T10:08:46.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.701 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.701 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2023-12-12T10:08:46.701 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.701 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.701 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2023-12-12T10:08:46.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.702 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.702 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2023-12-12T10:08:46.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2023-12-12T10:08:46.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2023-12-12T10:08:46.703 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.703 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.703 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.703 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2023-12-12T10:08:46.703 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.704 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.704 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.704 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2023-12-12T10:08:46.704 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.704 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.704 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.705 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2023-12-12T10:08:46.705 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2023-12-12T10:08:46.705 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.705 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.705 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.705 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.705 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.706 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.706 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.706 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.706 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.707 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2023-12-12T10:08:46.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2023-12-12T10:08:46.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.707 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.707 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.707 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.708 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.708 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-12T10:08:46.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-12T10:08:46.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.708 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.709 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.709 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.709 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.709 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.710 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.711 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.711 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.711 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.712 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.712 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.713 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.713 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.713 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.714 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.714 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.714 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.714 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.714 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.714 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.714 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.714 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.715 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.715 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.715 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.715 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.715 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.715 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.715 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.715 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.716 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.716 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.716 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.717 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.717 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.717 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.717 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.717 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.717 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.718 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2023-12-12T10:08:46.718 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2023-12-12T10:08:46.718 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.718 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.718 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.718 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.718 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.718 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.719 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.719 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.719 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.719 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.719 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-12T10:08:46.720 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-12T10:08:46.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.720 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2023-12-12T10:08:46.720 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.720 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.721 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.721 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.721 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.721 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.721 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.721 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2023-12-12T10:08:46.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2023-12-12T10:08:46.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:46.722 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.722 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2023-12-12T10:08:46.722 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.722 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.722 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.722 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.723 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.723 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2023-12-12T10:08:46.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2023-12-12T10:08:46.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.723 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2023-12-12T10:08:46.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-12T10:08:46.724 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.725 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.725 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2023-12-12T10:08:46.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2023-12-12T10:08:46.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.725 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.725 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.725 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2023-12-12T10:08:46.726 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.726 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.726 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-12T10:08:46.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.726 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.727 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.727 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2023-12-12T10:08:46.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2023-12-12T10:08:46.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.727 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2023-12-12T10:08:46.728 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.728 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.728 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.728 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.728 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.729 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2023-12-12T10:08:46.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2023-12-12T10:08:46.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.729 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2023-12-12T10:08:46.730 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.730 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.730 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2023-12-12T10:08:46.730 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2023-12-12T10:08:46.730 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.730 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.731 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.731 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2023-12-12T10:08:46.731 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2023-12-12T10:08:46.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.732 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2023-12-12T10:08:46.732 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2023-12-12T10:08:46.732 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2023-12-12T10:08:46.732 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.732 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.733 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.733 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2023-12-12T10:08:46.733 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.734 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.734 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.734 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.734 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.734 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.735 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.735 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.735 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.735 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.735 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.735 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.735 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.736 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.736 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.736 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.736 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.737 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.737 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.738 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.738 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.738 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.739 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.739 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.739 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.740 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.740 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.740 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.740 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.740 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.740 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.741 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.741 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.741 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.741 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.741 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.741 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.742 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.742 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.742 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.743 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.743 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.743 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2023-12-12T10:08:46.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2023-12-12T10:08:46.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.744 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.744 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.744 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2023-12-12T10:08:46.745 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.745 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.745 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.745 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.746 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2023-12-12T10:08:46.746 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2023-12-12T10:08:46.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.746 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.747 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.747 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.747 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.747 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2023-12-12T10:08:46.748 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2023-12-12T10:08:46.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.748 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.749 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.749 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.749 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.750 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.750 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2023-12-12T10:08:46.750 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.750 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.750 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2023-12-12T10:08:46.750 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2023-12-12T10:08:46.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.751 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.751 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.751 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.751 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.751 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.751 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.752 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.752 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2023-12-12T10:08:46.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2023-12-12T10:08:46.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.753 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.753 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.753 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.753 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.754 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.754 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.755 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.755 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.755 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.755 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.755 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.756 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.756 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.756 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.756 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.756 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.758 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.758 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.758 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.759 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.759 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.760 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.760 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.760 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.760 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.760 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.760 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.760 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.761 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.761 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.762 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.762 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.762 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.763 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.763 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2023-12-12T10:08:46.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2023-12-12T10:08:46.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.763 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.764 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.764 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.764 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.764 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.764 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.764 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.765 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2023-12-12T10:08:46.765 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2023-12-12T10:08:46.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.765 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.765 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.765 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.765 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.766 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.766 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.766 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.766 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.767 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2023-12-12T10:08:46.767 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.767 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.767 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.767 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.768 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.768 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.768 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.768 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.769 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.769 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.769 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.769 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.769 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.769 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.769 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.770 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.770 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.770 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.770 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.771 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.771 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.771 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.771 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.771 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.771 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.772 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.772 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.772 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.772 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.773 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.773 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.773 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.773 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.773 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.773 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.773 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.773 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.774 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.774 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.774 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.774 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.774 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.775 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.775 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.775 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.775 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.775 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.776 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.776 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.776 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.776 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.776 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.777 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2023-12-12T10:08:46.777 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.777 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.777 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.777 INFO:teuthology.orchestra.run.smithi042.stdout: "drive_group_interval": { 2023-12-12T10:08:46.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "drive_group_interval", 2023-12-12T10:08:46.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:46.778 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.778 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300.0", 2023-12-12T10:08:46.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.778 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-12T10:08:46.779 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.779 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.779 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.779 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.779 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.780 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.780 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.780 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.780 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.780 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.781 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.781 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.781 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.781 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.781 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.781 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.781 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.782 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.782 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.782 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.783 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.783 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.783 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.783 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.784 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.784 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.784 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.784 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.784 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.784 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.784 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.785 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.785 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.785 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.785 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.785 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.785 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.785 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.786 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.786 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.786 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.786 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.787 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.787 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.787 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.787 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.787 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.788 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.788 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2023-12-12T10:08:46.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2023-12-12T10:08:46.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.788 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.789 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2023-12-12T10:08:46.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.789 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-12T10:08:46.789 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.790 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.790 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2023-12-12T10:08:46.790 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.791 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.791 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.791 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.791 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.791 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.792 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.792 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.792 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.792 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.792 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.793 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.793 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.793 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.793 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.793 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.794 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.794 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.794 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.794 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.794 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.795 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.795 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.795 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.795 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.795 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.795 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.796 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.796 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.796 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.796 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.796 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.797 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.797 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.797 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.797 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.797 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.797 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.797 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.797 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.797 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.798 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.798 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.798 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.798 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.798 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.799 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.799 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.799 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.799 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.799 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.799 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.799 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.799 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.800 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2023-12-12T10:08:46.800 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2023-12-12T10:08:46.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.800 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.800 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.800 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.801 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.801 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.801 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.802 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2023-12-12T10:08:46.802 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2023-12-12T10:08:46.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.802 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.802 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2023-12-12T10:08:46.802 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.802 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.803 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.803 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.803 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.803 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.803 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2023-12-12T10:08:46.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2023-12-12T10:08:46.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.804 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.804 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.804 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.805 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.805 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2023-12-12T10:08:46.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2023-12-12T10:08:46.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.805 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.806 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.806 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.806 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.806 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.806 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.806 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.807 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.807 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2023-12-12T10:08:46.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2023-12-12T10:08:46.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2023-12-12T10:08:46.807 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.808 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.808 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.808 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.808 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.809 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.809 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2023-12-12T10:08:46.809 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2023-12-12T10:08:46.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.809 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.809 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.810 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.810 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.810 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.810 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.810 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.811 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2023-12-12T10:08:46.811 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2023-12-12T10:08:46.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.811 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.811 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.811 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.812 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.812 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.812 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.812 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.812 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.812 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.813 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2023-12-12T10:08:46.813 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2023-12-12T10:08:46.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.813 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.813 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.813 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.814 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.814 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.814 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2023-12-12T10:08:46.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2023-12-12T10:08:46.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.815 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.815 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2023-12-12T10:08:46.815 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2023-12-12T10:08:46.815 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.815 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.816 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.816 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.816 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2023-12-12T10:08:46.816 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2023-12-12T10:08:46.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.817 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.817 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.817 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.817 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.817 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.817 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.818 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.818 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2023-12-12T10:08:46.818 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2023-12-12T10:08:46.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.818 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.819 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.819 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.819 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.819 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.819 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.819 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.819 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.820 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2023-12-12T10:08:46.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2023-12-12T10:08:46.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.820 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.820 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.820 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.821 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.821 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.821 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.821 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.821 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.821 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.822 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.822 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.822 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2023-12-12T10:08:46.822 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.822 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.822 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.822 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2023-12-12T10:08:46.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2023-12-12T10:08:46.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.823 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.823 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.823 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.824 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.824 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2023-12-12T10:08:46.824 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2023-12-12T10:08:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2023-12-12T10:08:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2023-12-12T10:08:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.828 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.832 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2023-12-12T10:08:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.836 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.837 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.838 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.840 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.841 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.842 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.843 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.845 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.846 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2023-12-12T10:08:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.848 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.849 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.850 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.853 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2023-12-12T10:08:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2023-12-12T10:08:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2023-12-12T10:08:46.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-12T10:08:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.858 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.859 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2023-12-12T10:08:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.861 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.862 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.864 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.866 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.867 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.868 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.869 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2023-12-12T10:08:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.870 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2023-12-12T10:08:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2023-12-12T10:08:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.871 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-12T10:08:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2023-12-12T10:08:46.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2023-12-12T10:08:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-12T10:08:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2023-12-12T10:08:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2023-12-12T10:08:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.874 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2023-12-12T10:08:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-12T10:08:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2023-12-12T10:08:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2023-12-12T10:08:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.876 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-12T10:08:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2023-12-12T10:08:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2023-12-12T10:08:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.878 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-12T10:08:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.879 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2023-12-12T10:08:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2023-12-12T10:08:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.880 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2023-12-12T10:08:46.881 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2023-12-12T10:08:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2023-12-12T10:08:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2023-12-12T10:08:46.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-12T10:08:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.884 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2023-12-12T10:08:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2023-12-12T10:08:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.886 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.887 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2023-12-12T10:08:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2023-12-12T10:08:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.888 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2023-12-12T10:08:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2023-12-12T10:08:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.889 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2023-12-12T10:08:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.890 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2023-12-12T10:08:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2023-12-12T10:08:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.891 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.892 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2023-12-12T10:08:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2023-12-12T10:08:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.893 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.894 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.895 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.896 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.898 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.900 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.902 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.903 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2023-12-12T10:08:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2023-12-12T10:08:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.904 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2023-12-12T10:08:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2023-12-12T10:08:46.905 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.906 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2023-12-12T10:08:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2023-12-12T10:08:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-12T10:08:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.908 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2023-12-12T10:08:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.909 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.910 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.911 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.913 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.914 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.915 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.917 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.918 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.919 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2023-12-12T10:08:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.921 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.922 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.923 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.924 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.925 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.927 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2023-12-12T10:08:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2023-12-12T10:08:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2023-12-12T10:08:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.931 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-12T10:08:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.932 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2023-12-12T10:08:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2023-12-12T10:08:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2023-12-12T10:08:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2023-12-12T10:08:46.934 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2023-12-12T10:08:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2023-12-12T10:08:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.935 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-12T10:08:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2023-12-12T10:08:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2023-12-12T10:08:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2023-12-12T10:08:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2023-12-12T10:08:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2023-12-12T10:08:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2023-12-12T10:08:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2023-12-12T10:08:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2023-12-12T10:08:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.938 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2023-12-12T10:08:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2023-12-12T10:08:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.939 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.940 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2023-12-12T10:08:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2023-12-12T10:08:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2023-12-12T10:08:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2023-12-12T10:08:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.941 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2023-12-12T10:08:46.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2023-12-12T10:08:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.943 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.944 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2023-12-12T10:08:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2023-12-12T10:08:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.945 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.946 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2023-12-12T10:08:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2023-12-12T10:08:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.947 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2023-12-12T10:08:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2023-12-12T10:08:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2023-12-12T10:08:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2023-12-12T10:08:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2023-12-12T10:08:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2023-12-12T10:08:46.948 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2023-12-12T10:08:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2023-12-12T10:08:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.949 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2023-12-12T10:08:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2023-12-12T10:08:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2023-12-12T10:08:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2023-12-12T10:08:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2023-12-12T10:08:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2023-12-12T10:08:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2023-12-12T10:08:46.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2023-12-12T10:08:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.952 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2023-12-12T10:08:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2023-12-12T10:08:46.953 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2023-12-12T10:08:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2023-12-12T10:08:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.954 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2023-12-12T10:08:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2023-12-12T10:08:46.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2023-12-12T10:08:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2023-12-12T10:08:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2023-12-12T10:08:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-12T10:08:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2023-12-12T10:08:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2023-12-12T10:08:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2023-12-12T10:08:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2023-12-12T10:08:46.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2023-12-12T10:08:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2023-12-12T10:08:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2023-12-12T10:08:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.957 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2023-12-12T10:08:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard": "https://172.21.15.42:8443/" 2023-12-12T10:08:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: "always_on_modules": { 2023-12-12T10:08:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: "octopus": [ 2023-12-12T10:08:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2023-12-12T10:08:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2023-12-12T10:08:46.958 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2023-12-12T10:08:46.959 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2023-12-12T10:08:46.959 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2023-12-12T10:08:46.959 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2023-12-12T10:08:46.959 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2023-12-12T10:08:46.959 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2023-12-12T10:08:46.959 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2023-12-12T10:08:46.959 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2023-12-12T10:08:46.959 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.960 INFO:teuthology.orchestra.run.smithi042.stdout: "pacific": [ 2023-12-12T10:08:46.960 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2023-12-12T10:08:46.960 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2023-12-12T10:08:46.960 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2023-12-12T10:08:46.960 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2023-12-12T10:08:46.960 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2023-12-12T10:08:46.960 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2023-12-12T10:08:46.961 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2023-12-12T10:08:46.961 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2023-12-12T10:08:46.961 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2023-12-12T10:08:46.961 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2023-12-12T10:08:46.961 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.961 INFO:teuthology.orchestra.run.smithi042.stdout: "quincy": [ 2023-12-12T10:08:46.961 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2023-12-12T10:08:46.961 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2023-12-12T10:08:46.962 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2023-12-12T10:08:46.962 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2023-12-12T10:08:46.962 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2023-12-12T10:08:46.962 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2023-12-12T10:08:46.962 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2023-12-12T10:08:46.962 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2023-12-12T10:08:46.962 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2023-12-12T10:08:46.963 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2023-12-12T10:08:46.963 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2023-12-12T10:08:46.963 INFO:teuthology.orchestra.run.smithi042.stdout: "reef": [ 2023-12-12T10:08:46.963 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2023-12-12T10:08:46.963 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2023-12-12T10:08:46.963 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2023-12-12T10:08:46.963 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2023-12-12T10:08:46.963 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2023-12-12T10:08:46.964 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2023-12-12T10:08:46.964 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2023-12-12T10:08:46.964 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2023-12-12T10:08:46.964 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2023-12-12T10:08:46.964 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2023-12-12T10:08:46.964 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:46.964 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.964 INFO:teuthology.orchestra.run.smithi042.stdout: "last_failure_osd_epoch": 92, 2023-12-12T10:08:46.965 INFO:teuthology.orchestra.run.smithi042.stdout: "active_clients": [ 2023-12-12T10:08:46.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2023-12-12T10:08:46.965 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2023-12-12T10:08:46.965 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2023-12-12T10:08:46.965 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2023-12-12T10:08:46.966 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 2178181485 2023-12-12T10:08:46.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.966 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:46.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.966 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "libcephsqlite", 2023-12-12T10:08:46.966 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2023-12-12T10:08:46.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.967 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2023-12-12T10:08:46.967 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2023-12-12T10:08:46.967 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1027407715 2023-12-12T10:08:46.967 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.967 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:46.967 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.967 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.967 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2023-12-12T10:08:46.968 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2023-12-12T10:08:46.968 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.968 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2023-12-12T10:08:46.968 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2023-12-12T10:08:46.968 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 41263368 2023-12-12T10:08:46.968 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.968 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:46.968 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.969 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2023-12-12T10:08:46.969 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2023-12-12T10:08:46.969 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2023-12-12T10:08:46.969 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2023-12-12T10:08:46.969 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 362240647 2023-12-12T10:08:46.969 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.970 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:46.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2023-12-12T10:08:46.970 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2023-12-12T10:08:46.970 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2023-12-12T10:08:46.970 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2023-12-12T10:08:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1105092836 2023-12-12T10:08:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2023-12-12T10:08:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2023-12-12T10:08:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: { 2023-12-12T10:08:46.971 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2023-12-12T10:08:46.972 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2023-12-12T10:08:46.972 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1695152089 2023-12-12T10:08:46.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.972 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:46.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:46.972 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2023-12-12T10:08:46.972 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T10:08:46.974 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:44 smithi173 ceph-mon[103308]: pgmap v311: 129 pgs: 13 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 72 active+clean; 717 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:46.975 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:44 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/273411111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T10:08:46.975 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:45 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/4261012708' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-12T10:08:46.975 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:44 smithi111 ceph-mon[102565]: pgmap v311: 129 pgs: 13 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 72 active+clean; 717 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:46.975 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:44 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/273411111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T10:08:46.975 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:45 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/4261012708' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-12T10:08:46.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fbf040a17c0 msgr2=0x7fbf040a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:46.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fbf040a17c0 0x7fbf040a3c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009730 tx=0x7fbf0c006cb0 comp rx=0 tx=0).stop 2023-12-12T10:08:46.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c10cfa0 msgr2=0x7fbf1c19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:08:46.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c10cfa0 0x7fbf1c19a170 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400ef30 tx=0x7fbf1400c450 comp rx=0 tx=0).stop 2023-12-12T10:08:46.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 shutdown_connections 2023-12-12T10:08:46.976 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fbf040a17c0 0x7fbf040a3c80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:46.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf1c10cfa0 0x7fbf1c19a170 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:46.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf1c100580 0x7fbf1c10c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:46.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 --2- 172.21.15.42:0/4261012708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbf1c1056a0 0x7fbf1c10c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:08:46.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 >> 172.21.15.42:0/4261012708 conn(0x7fbf1c0fbe40 msgr2=0x7fbf1c0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:08:46.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 shutdown_connections 2023-12-12T10:08:46.977 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:08:44.644+0000 7fbf2131c700 1 -- 172.21.15.42:0/4261012708 wait complete. 2023-12-12T10:08:46.986 DEBUG:tasks.check_counter:Getting stats from a 2023-12-12T10:08:46.986 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.a perf dump 2023-12-12T10:08:46.987 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:44 smithi042 ceph-mon[98894]: pgmap v311: 129 pgs: 13 active+clean+snaptrim, 44 active+clean+snaptrim_wait, 72 active+clean; 717 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2023-12-12T10:08:46.987 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:44 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/273411111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T10:08:46.987 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:45 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/4261012708' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-12T10:08:46.987 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:46 smithi042 ceph-mon[98894]: pgmap v312: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 16 op/s 2023-12-12T10:08:46.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:46 smithi111 ceph-mon[102565]: pgmap v312: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 16 op/s 2023-12-12T10:08:47.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:46 smithi173 ceph-mon[103308]: pgmap v312: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 16 op/s 2023-12-12T10:08:47.638 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T10:08:48.888 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:48 smithi042 ceph-mon[98894]: pgmap v313: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-12T10:08:48.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:48 smithi111 ceph-mon[102565]: pgmap v313: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-12T10:08:49.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:48 smithi173 ceph-mon[103308]: pgmap v313: 129 pgs: 5 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 117 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-12T10:08:50.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:50 smithi042 ceph-mon[98894]: pgmap v314: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 16 op/s 2023-12-12T10:08:50.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:50 smithi111 ceph-mon[102565]: pgmap v314: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 16 op/s 2023-12-12T10:08:51.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:50 smithi173 ceph-mon[103308]: pgmap v314: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 0 B/s wr, 16 op/s 2023-12-12T10:08:52.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:52 smithi042 ceph-mon[98894]: pgmap v315: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-12T10:08:52.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:52 smithi111 ceph-mon[102565]: pgmap v315: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-12T10:08:53.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:52 smithi173 ceph-mon[103308]: pgmap v315: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-12T10:08:54.316 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T10:08:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:08:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 1231, 2023-12-12T10:08:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 1231, 2023-12-12T10:08:54.316 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 67482786, 2023-12-12T10:08:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 382046, 2023-12-12T10:08:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 8, 2023-12-12T10:08:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 4, 2023-12-12T10:08:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.160925467, 2023-12-12T10:08:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.035188472, 2023-12-12T10:08:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.069519504, 2023-12-12T10:08:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.043796117, 2023-12-12T10:08:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:08:54.317 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 1230, 2023-12-12T10:08:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.234696694, 2023-12-12T10:08:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000190810 2023-12-12T10:08:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:08:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.318 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-12T10:08:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2023-12-12T10:08:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:08:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 480, 2023-12-12T10:08:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 446, 2023-12-12T10:08:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 60389363, 2023-12-12T10:08:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 194455, 2023-12-12T10:08:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 6, 2023-12-12T10:08:54.319 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 5, 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.102919505, 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.018015875, 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.079097193, 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.014030234, 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 445, 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.049976651, 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000112307 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.320 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:08:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1599136, 2023-12-12T10:08:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 123168 2023-12-12T10:08:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:08:54.321 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 874, 2023-12-12T10:08:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 1025, 2023-12-12T10:08:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 49388199, 2023-12-12T10:08:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 691672, 2023-12-12T10:08:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 8, 2023-12-12T10:08:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 6, 2023-12-12T10:08:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.115205766, 2023-12-12T10:08:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.029749153, 2023-12-12T10:08:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.047210657, 2023-12-12T10:08:54.322 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.029348484, 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 1022, 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.079369306, 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000077660 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-12T10:08:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 420480 2023-12-12T10:08:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2023-12-12T10:08:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 1, 2023-12-12T10:08:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2023-12-12T10:08:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-MDSRank": { 2023-12-12T10:08:54.324 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2023-12-12T10:08:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2023-12-12T10:08:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3763, 2023-12-12T10:08:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.117386159, 2023-12-12T10:08:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000031194 2023-12-12T10:08:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.325 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-PurgeQueue": { 2023-12-12T10:08:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2023-12-12T10:08:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2023-12-12T10:08:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2023-12-12T10:08:54.326 INFO:teuthology.orchestra.run.smithi042.stdout: "request": 0, 2023-12-12T10:08:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "reply": 0, 2023-12-12T10:08:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "reply_latency": { 2023-12-12T10:08:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_reply": 0, 2023-12-12T10:08:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "forward": 0, 2023-12-12T10:08:54.327 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_complete": 0, 2023-12-12T10:08:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_keys": 0, 2023-12-12T10:08:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_commit": 0, 2023-12-12T10:08:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_split": 0, 2023-12-12T10:08:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_merge": 0, 2023-12-12T10:08:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes": 32310, 2023-12-12T10:08:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_top": 0, 2023-12-12T10:08:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_bottom": 32310, 2023-12-12T10:08:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pin_tail": 0, 2023-12-12T10:08:54.328 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pinned": 32310, 2023-12-12T10:08:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_expired": 0, 2023-12-12T10:08:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_with_caps": 0, 2023-12-12T10:08:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": 0, 2023-12-12T10:08:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "subtrees": 1, 2023-12-12T10:08:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse": 0, 2023-12-12T10:08:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_hit": 0, 2023-12-12T10:08:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_forward": 0, 2023-12-12T10:08:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_discover": 0, 2023-12-12T10:08:54.329 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_dir_fetch": 0, 2023-12-12T10:08:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_remote_ino": 0, 2023-12-12T10:08:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_lock": 0, 2023-12-12T10:08:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "load_cent": 0, 2023-12-12T10:08:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "q": 0, 2023-12-12T10:08:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "exported": 0, 2023-12-12T10:08:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "exported_inodes": 0, 2023-12-12T10:08:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "imported": 0, 2023-12-12T10:08:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "imported_inodes": 0, 2023-12-12T10:08:54.330 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_dir_fetch": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_peer_discover": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rfiles": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rbytes": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rsnaps": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_set_tag": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:08:54.331 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:08:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:08:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:08:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_file_inodes": 0, 2023-12-12T10:08:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_inode_file_caps": 0, 2023-12-12T10:08:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_revoke": 0, 2023-12-12T10:08:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_grant": 0, 2023-12-12T10:08:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_trunc": 0, 2023-12-12T10:08:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-12T10:08:54.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:08:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps": 0, 2023-12-12T10:08:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps_dirty": 0, 2023-12-12T10:08:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_cap_release": 0, 2023-12-12T10:08:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "process_request_cap_release": 0 2023-12-12T10:08:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache": { 2023-12-12T10:08:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update": 0, 2023-12-12T10:08:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update_receipt": 0, 2023-12-12T10:08:54.333 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_try_discover": 0, 2023-12-12T10:08:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_send_discover": 0, 2023-12-12T10:08:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_handle_discover": 0, 2023-12-12T10:08:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays": 0, 2023-12-12T10:08:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_delayed": 0, 2023-12-12T10:08:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:08:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_created": 0, 2023-12-12T10:08:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_enqueued": 0, 2023-12-12T10:08:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_reintegrated": 0, 2023-12-12T10:08:54.334 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_migrated": 0, 2023-12-12T10:08:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_processing": 0, 2023-12-12T10:08:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:08:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:08:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_started": 0, 2023-12-12T10:08:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_completed": 0, 2023-12-12T10:08:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:08:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_exportdir": 0, 2023-12-12T10:08:54.335 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_flush": 0, 2023-12-12T10:08:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragmentdir": 0, 2023-12-12T10:08:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragstats": 0, 2023-12-12T10:08:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_inodestats": 0 2023-12-12T10:08:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log": { 2023-12-12T10:08:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "evlrg": 0, 2023-12-12T10:08:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "evadd": 0, 2023-12-12T10:08:54.336 INFO:teuthology.orchestra.run.smithi042.stdout: "evex": 0, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "evtrm": 0, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "ev": 65321, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "evexg": 0, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "evexd": 0, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "segadd": 0, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "segex": 0, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "segtrm": 0, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "seg": 91, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "segmjr": 7, 2023-12-12T10:08:54.337 INFO:teuthology.orchestra.run.smithi042.stdout: "segexg": 0, 2023-12-12T10:08:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "segexd": 0, 2023-12-12T10:08:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "expos": 4194304, 2023-12-12T10:08:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "wrpos": 179188894, 2023-12-12T10:08:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "rdpos": 179184537, 2023-12-12T10:08:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "jlat": { 2023-12-12T10:08:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.338 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "replayed": 65321 2023-12-12T10:08:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_mem": { 2023-12-12T10:08:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 11944, 2023-12-12T10:08:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "ino+": 69750, 2023-12-12T10:08:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "ino-": 57806, 2023-12-12T10:08:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "dir": 523, 2023-12-12T10:08:54.339 INFO:teuthology.orchestra.run.smithi042.stdout: "dir+": 2381, 2023-12-12T10:08:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "dir-": 1858, 2023-12-12T10:08:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "dn": 32310, 2023-12-12T10:08:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "dn+": 114073, 2023-12-12T10:08:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "dn-": 81763, 2023-12-12T10:08:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "cap": 0, 2023-12-12T10:08:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "cap+": 0, 2023-12-12T10:08:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "cap-": 0, 2023-12-12T10:08:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "rss": 130336, 2023-12-12T10:08:54.340 INFO:teuthology.orchestra.run.smithi042.stdout: "heap": 207228 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_server": { 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_client_request": 0, 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_server_request": 0, 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_request": 0, 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_session": 0, 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_peer_request": 0, 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "req_create_latency": { 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.341 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getattr_latency": { 2023-12-12T10:08:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.342 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getfilelock_latency": { 2023-12-12T10:08:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "req_link_latency": { 2023-12-12T10:08:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookup_latency": { 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookuphash_latency": { 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupino_latency": { 2023-12-12T10:08:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupname_latency": { 2023-12-12T10:08:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.345 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupparent_latency": { 2023-12-12T10:08:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupsnap_latency": { 2023-12-12T10:08:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.346 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lssnap_latency": { 2023-12-12T10:08:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mkdir_latency": { 2023-12-12T10:08:54.347 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mknod_latency": { 2023-12-12T10:08:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.348 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mksnap_latency": { 2023-12-12T10:08:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "req_open_latency": { 2023-12-12T10:08:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "req_readdir_latency": { 2023-12-12T10:08:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rename_latency": { 2023-12-12T10:08:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.350 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "req_renamesnap_latency": { 2023-12-12T10:08:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "req_snapdiff_latency": { 2023-12-12T10:08:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.351 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmdir_latency": { 2023-12-12T10:08:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmsnap_latency": { 2023-12-12T10:08:54.352 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmxattr_latency": { 2023-12-12T10:08:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.353 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setattr_latency": { 2023-12-12T10:08:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setdirlayout_latency": { 2023-12-12T10:08:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setfilelock_latency": { 2023-12-12T10:08:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setlayout_latency": { 2023-12-12T10:08:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.355 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setxattr_latency": { 2023-12-12T10:08:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "req_symlink_latency": { 2023-12-12T10:08:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.356 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "req_unlink_latency": { 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:08:54.357 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getvxattr_latency": { 2023-12-12T10:08:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sessions": { 2023-12-12T10:08:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "session_count": 2, 2023-12-12T10:08:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "session_add": 4, 2023-12-12T10:08:54.358 INFO:teuthology.orchestra.run.smithi042.stdout: "session_remove": 2, 2023-12-12T10:08:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_open": 4, 2023-12-12T10:08:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_stale": 0, 2023-12-12T10:08:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "total_load": 0, 2023-12-12T10:08:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "average_load": 0, 2023-12-12T10:08:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "avg_session_uptime": 3614, 2023-12-12T10:08:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "mdthresh_evicted": 0 2023-12-12T10:08:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.359 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2023-12-12T10:08:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:08:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2023-12-12T10:08:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:08:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:08:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:08:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:08:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:08:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:08:54.360 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2023-12-12T10:08:54.361 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:08:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:08:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:08:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:08:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:08:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:08:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:08:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2023-12-12T10:08:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:08:54.362 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:08:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:08:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2023-12-12T10:08:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2023-12-12T10:08:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2023-12-12T10:08:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:08:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:08:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:08:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:08:54.363 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 645238, 2023-12-12T10:08:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 12014, 2023-12-12T10:08:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2023-12-12T10:08:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2023-12-12T10:08:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2023-12-12T10:08:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2023-12-12T10:08:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:08:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2023-12-12T10:08:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:08:54.364 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2023-12-12T10:08:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 9848, 2023-12-12T10:08:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 190, 2023-12-12T10:08:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:08:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:08:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2023-12-12T10:08:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2023-12-12T10:08:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 50292417, 2023-12-12T10:08:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 396854, 2023-12-12T10:08:54.365 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2023-12-12T10:08:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2023-12-12T10:08:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2023-12-12T10:08:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2023-12-12T10:08:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": { 2023-12-12T10:08:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2023-12-12T10:08:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2023-12-12T10:08:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 2328, 2023-12-12T10:08:54.366 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 21, 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 3, 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 2328, 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 2328, 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 5.421956278, 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.002329019 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2023-12-12T10:08:54.367 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 2328, 2023-12-12T10:08:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 2329 2023-12-12T10:08:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 2328, 2023-12-12T10:08:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 2328, 2023-12-12T10:08:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2023-12-12T10:08:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2023-12-12T10:08:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2023-12-12T10:08:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 1475, 2023-12-12T10:08:54.368 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2023-12-12T10:08:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 851, 2023-12-12T10:08:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2023-12-12T10:08:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2023-12-12T10:08:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2023-12-12T10:08:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2023-12-12T10:08:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2023-12-12T10:08:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2023-12-12T10:08:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2023-12-12T10:08:54.369 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2023-12-12T10:08:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2023-12-12T10:08:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2023-12-12T10:08:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 1, 2023-12-12T10:08:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2023-12-12T10:08:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:08:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2023-12-12T10:08:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:08:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2023-12-12T10:08:54.370 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2023-12-12T10:08:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2023-12-12T10:08:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:08:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2023-12-12T10:08:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:08:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2023-12-12T10:08:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2023-12-12T10:08:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2023-12-12T10:08:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2023-12-12T10:08:54.371 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2023-12-12T10:08:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2023-12-12T10:08:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2023-12-12T10:08:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2023-12-12T10:08:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2023-12-12T10:08:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2023-12-12T10:08:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2023-12-12T10:08:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2023-12-12T10:08:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2023-12-12T10:08:54.372 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 98, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 11, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 11, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 11, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2023-12-12T10:08:54.373 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2023-12-12T10:08:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2023-12-12T10:08:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 2, 2023-12-12T10:08:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2023-12-12T10:08:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "oft": { 2023-12-12T10:08:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_objs": 0, 2023-12-12T10:08:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:08:54.374 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_updates": 0, 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_removes": 0 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "osd-slow-ops": { 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_ops_count": 0 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "purge_queue": { 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops": 0, 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops_high_water": 0, 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing": 0, 2023-12-12T10:08:54.375 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_high_water": 0, 2023-12-12T10:08:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executed": 0, 2023-12-12T10:08:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_item_in_journal": 0 2023-12-12T10:08:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:08:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2023-12-12T10:08:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 2581, 2023-12-12T10:08:54.376 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 177012343, 2023-12-12T10:08:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 2581, 2023-12-12T10:08:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2023-12-12T10:08:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2023-12-12T10:08:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 2581, 2023-12-12T10:08:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 177012343, 2023-12-12T10:08:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.377 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": { 2023-12-12T10:08:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2023-12-12T10:08:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2023-12-12T10:08:54.378 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2023-12-12T10:08:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2023-12-12T10:08:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 2328, 2023-12-12T10:08:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 175006734, 2023-12-12T10:08:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 2328, 2023-12-12T10:08:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 175006734, 2023-12-12T10:08:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.379 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": { 2023-12-12T10:08:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2023-12-12T10:08:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2023-12-12T10:08:54.380 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2023-12-12T10:08:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2023-12-12T10:08:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 2328, 2023-12-12T10:08:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 2328, 2023-12-12T10:08:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 2328, 2023-12-12T10:08:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2328, 2023-12-12T10:08:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.381 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:08:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2023-12-12T10:08:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2023-12-12T10:08:54.382 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2023-12-12T10:08:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2023-12-12T10:08:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2023-12-12T10:08:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2023-12-12T10:08:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2023-12-12T10:08:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2023-12-12T10:08:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.383 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle-0x5562d3c08960": { 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2023-12-12T10:08:54.384 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2023-12-12T10:08:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2023-12-12T10:08:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2023-12-12T10:08:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2023-12-12T10:08:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2023-12-12T10:08:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:54.385 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:54.386 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:54.386 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T10:08:54.653 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:54 smithi042 ceph-mon[98894]: pgmap v316: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-12T10:08:54.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:54 smithi042 ceph-mon[98894]: osdmap e98: 12 total, 12 up, 12 in 2023-12-12T10:08:54.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:54 smithi111 ceph-mon[102565]: pgmap v316: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-12T10:08:54.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:54 smithi111 ceph-mon[102565]: osdmap e98: 12 total, 12 up, 12 in 2023-12-12T10:08:54.822 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2023-12-12T10:08:54.822 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2023-12-12T10:08:54.822 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2023-12-12T10:08:54.822 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2023-12-12T10:08:54.822 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2023-12-12T10:08:54.822 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2023-12-12T10:08:54.822 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2023-12-12T10:08:54.823 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2023-12-12T10:08:54.823 DEBUG:tasks.check_counter:Getting stats from d 2023-12-12T10:08:54.823 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.d perf dump 2023-12-12T10:08:54.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:54 smithi173 ceph-mon[103308]: pgmap v316: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 11 op/s 2023-12-12T10:08:54.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:54 smithi173 ceph-mon[103308]: osdmap e98: 12 total, 12 up, 12 in 2023-12-12T10:08:55.423 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T10:08:56.745 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T10:08:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:08:56.745 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2023-12-12T10:08:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2023-12-12T10:08:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2992, 2023-12-12T10:08:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2023-12-12T10:08:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2023-12-12T10:08:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2023-12-12T10:08:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.002264749, 2023-12-12T10:08:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000050837, 2023-12-12T10:08:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000781606, 2023-12-12T10:08:56.746 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000831404, 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000277134 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:56.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-12T10:08:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2023-12-12T10:08:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:08:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2023-12-12T10:08:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 156, 2023-12-12T10:08:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2023-12-12T10:08:56.748 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 181264, 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.013584994, 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.009108107, 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.002060798, 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 153, 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.019688222, 2023-12-12T10:08:56.749 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000128681 2023-12-12T10:08:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:08:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-12T10:08:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 181264 2023-12-12T10:08:56.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:08:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 215, 2023-12-12T10:08:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 205, 2023-12-12T10:08:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1422832, 2023-12-12T10:08:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 122896, 2023-12-12T10:08:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2023-12-12T10:08:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2023-12-12T10:08:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.041971881, 2023-12-12T10:08:56.751 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.013761758, 2023-12-12T10:08:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.032331585, 2023-12-12T10:08:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.004899852, 2023-12-12T10:08:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:08:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 204, 2023-12-12T10:08:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.018904132, 2023-12-12T10:08:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000092667 2023-12-12T10:08:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:08:56.752 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1422832, 2023-12-12T10:08:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 122896 2023-12-12T10:08:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2023-12-12T10:08:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2023-12-12T10:08:56.753 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2023-12-12T10:08:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2023-12-12T10:08:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:08:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2023-12-12T10:08:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:08:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:08:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:08:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:08:56.754 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:08:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:08:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:08:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:08:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:08:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:08:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:08:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:08:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:08:56.755 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2023-12-12T10:08:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:08:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2023-12-12T10:08:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:08:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:08:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:08:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:08:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:08:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:08:56.756 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:08:56.757 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:08:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:08:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 149007, 2023-12-12T10:08:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2023-12-12T10:08:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2023-12-12T10:08:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2023-12-12T10:08:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2023-12-12T10:08:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2023-12-12T10:08:56.758 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2023-12-12T10:08:56.759 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2023-12-12T10:08:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2023-12-12T10:08:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2023-12-12T10:08:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2023-12-12T10:08:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2023-12-12T10:08:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2023-12-12T10:08:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:08:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:56.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2023-12-12T10:08:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 217, 2023-12-12T10:08:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1392527, 2023-12-12T10:08:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 217, 2023-12-12T10:08:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2023-12-12T10:08:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2023-12-12T10:08:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 217, 2023-12-12T10:08:56.761 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1392527, 2023-12-12T10:08:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:56.762 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:56.762 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T10:08:56.763 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:56 smithi111 ceph-mon[102565]: pgmap v318: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:08:56.763 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:56 smithi042 ceph-mon[98894]: pgmap v318: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:08:56.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:56 smithi173 ceph-mon[103308]: pgmap v318: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:08:57.194 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-12-12T10:08:57.194 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-12-12T10:08:57.194 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-12-12T10:08:57.194 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-12-12T10:08:57.194 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-12-12T10:08:57.194 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-12-12T10:08:57.194 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-12-12T10:08:57.195 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-12-12T10:08:57.195 DEBUG:tasks.check_counter:Getting stats from g 2023-12-12T10:08:57.195 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.g perf dump 2023-12-12T10:08:57.795 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T10:08:58.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:08:58 smithi042 ceph-mon[98894]: pgmap v319: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:08:58.748 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:08:58 smithi111 ceph-mon[102565]: pgmap v319: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:08:58.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:08:58 smithi173 ceph-mon[103308]: pgmap v319: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:08:59.132 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T10:08:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:08:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 888, 2023-12-12T10:08:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 890, 2023-12-12T10:08:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 42249736, 2023-12-12T10:08:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 276169, 2023-12-12T10:08:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 8, 2023-12-12T10:08:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 4, 2023-12-12T10:08:59.133 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.125177038, 2023-12-12T10:08:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.034726628, 2023-12-12T10:08:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.050086016, 2023-12-12T10:08:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.030140930, 2023-12-12T10:08:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:08:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 888, 2023-12-12T10:08:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.087365949, 2023-12-12T10:08:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000098385 2023-12-12T10:08:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.134 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:08:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-12T10:08:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2023-12-12T10:08:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:08:59.135 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 1151, 2023-12-12T10:08:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 1117, 2023-12-12T10:08:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 43654970, 2023-12-12T10:08:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 403438, 2023-12-12T10:08:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2023-12-12T10:08:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 4, 2023-12-12T10:08:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.139576548, 2023-12-12T10:08:59.136 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.033163630, 2023-12-12T10:08:59.137 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.078926589, 2023-12-12T10:08:59.137 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.031296450, 2023-12-12T10:08:59.137 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:08:59.137 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 1116, 2023-12-12T10:08:59.137 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.067464403, 2023-12-12T10:08:59.137 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000060451 2023-12-12T10:08:59.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.137 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:08:59.137 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.138 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.138 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.138 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1599408, 2023-12-12T10:08:59.138 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 123792 2023-12-12T10:08:59.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.138 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:08:59.138 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 442, 2023-12-12T10:08:59.139 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 593, 2023-12-12T10:08:59.139 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 37861823, 2023-12-12T10:08:59.139 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 560055, 2023-12-12T10:08:59.139 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 8, 2023-12-12T10:08:59.139 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 6, 2023-12-12T10:08:59.139 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.077216059, 2023-12-12T10:08:59.139 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.017965043, 2023-12-12T10:08:59.139 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.036889396, 2023-12-12T10:08:59.139 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.016469395, 2023-12-12T10:08:59.140 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:08:59.140 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 591, 2023-12-12T10:08:59.140 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.049824682, 2023-12-12T10:08:59.140 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000084305 2023-12-12T10:08:59.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.140 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:08:59.140 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.141 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.141 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.141 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-12-12T10:08:59.141 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 424448 2023-12-12T10:08:59.141 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.141 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2023-12-12T10:08:59.141 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 1, 2023-12-12T10:08:59.141 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2023-12-12T10:08:59.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.142 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-MDSRank": { 2023-12-12T10:08:59.142 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2023-12-12T10:08:59.142 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2023-12-12T10:08:59.142 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3677, 2023-12-12T10:08:59.142 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.103218543, 2023-12-12T10:08:59.142 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000028071 2023-12-12T10:08:59.142 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:59.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.143 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-PurgeQueue": { 2023-12-12T10:08:59.143 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2023-12-12T10:08:59.143 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2023-12-12T10:08:59.143 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.143 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.143 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.143 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:59.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.143 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2023-12-12T10:08:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: "request": 0, 2023-12-12T10:08:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: "reply": 0, 2023-12-12T10:08:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: "reply_latency": { 2023-12-12T10:08:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.144 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_reply": 0, 2023-12-12T10:08:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: "forward": 0, 2023-12-12T10:08:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_complete": 0, 2023-12-12T10:08:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_keys": 0, 2023-12-12T10:08:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_commit": 0, 2023-12-12T10:08:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_split": 0, 2023-12-12T10:08:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_merge": 0, 2023-12-12T10:08:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes": 10859, 2023-12-12T10:08:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_top": 0, 2023-12-12T10:08:59.145 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_bottom": 10859, 2023-12-12T10:08:59.146 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pin_tail": 0, 2023-12-12T10:08:59.146 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pinned": 10859, 2023-12-12T10:08:59.146 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_expired": 0, 2023-12-12T10:08:59.146 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_with_caps": 0, 2023-12-12T10:08:59.146 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": 0, 2023-12-12T10:08:59.146 INFO:teuthology.orchestra.run.smithi042.stdout: "subtrees": 1, 2023-12-12T10:08:59.146 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse": 0, 2023-12-12T10:08:59.146 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_hit": 0, 2023-12-12T10:08:59.147 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_forward": 0, 2023-12-12T10:08:59.147 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_discover": 0, 2023-12-12T10:08:59.147 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_dir_fetch": 0, 2023-12-12T10:08:59.147 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_remote_ino": 0, 2023-12-12T10:08:59.147 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_lock": 0, 2023-12-12T10:08:59.147 INFO:teuthology.orchestra.run.smithi042.stdout: "load_cent": 0, 2023-12-12T10:08:59.147 INFO:teuthology.orchestra.run.smithi042.stdout: "q": 0, 2023-12-12T10:08:59.147 INFO:teuthology.orchestra.run.smithi042.stdout: "exported": 0, 2023-12-12T10:08:59.147 INFO:teuthology.orchestra.run.smithi042.stdout: "exported_inodes": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "imported": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "imported_inodes": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_dir_fetch": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_peer_discover": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rfiles": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rbytes": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rsnaps": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:08:59.148 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_set_tag": 0, 2023-12-12T10:08:59.149 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:08:59.149 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:08:59.149 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:08:59.149 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:08:59.149 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:08:59.149 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_file_inodes": 0, 2023-12-12T10:08:59.149 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_inode_file_caps": 0, 2023-12-12T10:08:59.149 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_revoke": 0, 2023-12-12T10:08:59.149 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_grant": 0, 2023-12-12T10:08:59.150 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_trunc": 0, 2023-12-12T10:08:59.150 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-12T10:08:59.150 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:08:59.150 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps": 0, 2023-12-12T10:08:59.150 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps_dirty": 0, 2023-12-12T10:08:59.150 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_cap_release": 0, 2023-12-12T10:08:59.150 INFO:teuthology.orchestra.run.smithi042.stdout: "process_request_cap_release": 0 2023-12-12T10:08:59.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.151 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache": { 2023-12-12T10:08:59.151 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update": 0, 2023-12-12T10:08:59.151 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update_receipt": 0, 2023-12-12T10:08:59.151 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_try_discover": 0, 2023-12-12T10:08:59.151 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_send_discover": 0, 2023-12-12T10:08:59.151 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_handle_discover": 0, 2023-12-12T10:08:59.152 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays": 0, 2023-12-12T10:08:59.152 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_delayed": 0, 2023-12-12T10:08:59.152 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:08:59.152 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_created": 0, 2023-12-12T10:08:59.152 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_enqueued": 0, 2023-12-12T10:08:59.152 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_reintegrated": 0, 2023-12-12T10:08:59.152 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_migrated": 0, 2023-12-12T10:08:59.152 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_processing": 0, 2023-12-12T10:08:59.152 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:08:59.153 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:08:59.153 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_started": 0, 2023-12-12T10:08:59.153 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_completed": 0, 2023-12-12T10:08:59.153 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:08:59.153 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_exportdir": 0, 2023-12-12T10:08:59.153 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_flush": 0, 2023-12-12T10:08:59.153 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragmentdir": 0, 2023-12-12T10:08:59.153 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragstats": 0, 2023-12-12T10:08:59.153 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_inodestats": 0 2023-12-12T10:08:59.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.154 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log": { 2023-12-12T10:08:59.154 INFO:teuthology.orchestra.run.smithi042.stdout: "evlrg": 0, 2023-12-12T10:08:59.154 INFO:teuthology.orchestra.run.smithi042.stdout: "evadd": 0, 2023-12-12T10:08:59.154 INFO:teuthology.orchestra.run.smithi042.stdout: "evex": 0, 2023-12-12T10:08:59.154 INFO:teuthology.orchestra.run.smithi042.stdout: "evtrm": 0, 2023-12-12T10:08:59.155 INFO:teuthology.orchestra.run.smithi042.stdout: "ev": 47777, 2023-12-12T10:08:59.155 INFO:teuthology.orchestra.run.smithi042.stdout: "evexg": 0, 2023-12-12T10:08:59.155 INFO:teuthology.orchestra.run.smithi042.stdout: "evexd": 0, 2023-12-12T10:08:59.155 INFO:teuthology.orchestra.run.smithi042.stdout: "segadd": 0, 2023-12-12T10:08:59.155 INFO:teuthology.orchestra.run.smithi042.stdout: "segex": 0, 2023-12-12T10:08:59.155 INFO:teuthology.orchestra.run.smithi042.stdout: "segtrm": 0, 2023-12-12T10:08:59.155 INFO:teuthology.orchestra.run.smithi042.stdout: "seg": 67, 2023-12-12T10:08:59.155 INFO:teuthology.orchestra.run.smithi042.stdout: "segmjr": 5, 2023-12-12T10:08:59.155 INFO:teuthology.orchestra.run.smithi042.stdout: "segexg": 0, 2023-12-12T10:08:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: "segexd": 0, 2023-12-12T10:08:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: "expos": 4194304, 2023-12-12T10:08:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: "wrpos": 125706584, 2023-12-12T10:08:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: "rdpos": 125692891, 2023-12-12T10:08:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: "jlat": { 2023-12-12T10:08:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.157 INFO:teuthology.orchestra.run.smithi042.stdout: "replayed": 47777 2023-12-12T10:08:59.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.157 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_mem": { 2023-12-12T10:08:59.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 13, 2023-12-12T10:08:59.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ino+": 40684, 2023-12-12T10:08:59.157 INFO:teuthology.orchestra.run.smithi042.stdout: "ino-": 40671, 2023-12-12T10:08:59.157 INFO:teuthology.orchestra.run.smithi042.stdout: "dir": 11, 2023-12-12T10:08:59.157 INFO:teuthology.orchestra.run.smithi042.stdout: "dir+": 1431, 2023-12-12T10:08:59.157 INFO:teuthology.orchestra.run.smithi042.stdout: "dir-": 1420, 2023-12-12T10:08:59.158 INFO:teuthology.orchestra.run.smithi042.stdout: "dn": 10859, 2023-12-12T10:08:59.158 INFO:teuthology.orchestra.run.smithi042.stdout: "dn+": 64954, 2023-12-12T10:08:59.158 INFO:teuthology.orchestra.run.smithi042.stdout: "dn-": 54095, 2023-12-12T10:08:59.158 INFO:teuthology.orchestra.run.smithi042.stdout: "cap": 0, 2023-12-12T10:08:59.158 INFO:teuthology.orchestra.run.smithi042.stdout: "cap+": 0, 2023-12-12T10:08:59.158 INFO:teuthology.orchestra.run.smithi042.stdout: "cap-": 0, 2023-12-12T10:08:59.158 INFO:teuthology.orchestra.run.smithi042.stdout: "rss": 84072, 2023-12-12T10:08:59.158 INFO:teuthology.orchestra.run.smithi042.stdout: "heap": 207228 2023-12-12T10:08:59.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.159 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_server": { 2023-12-12T10:08:59.159 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_client_request": 0, 2023-12-12T10:08:59.159 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_server_request": 0, 2023-12-12T10:08:59.159 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_request": 0, 2023-12-12T10:08:59.159 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_session": 0, 2023-12-12T10:08:59.159 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_peer_request": 0, 2023-12-12T10:08:59.159 INFO:teuthology.orchestra.run.smithi042.stdout: "req_create_latency": { 2023-12-12T10:08:59.159 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.160 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.160 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.160 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getattr_latency": { 2023-12-12T10:08:59.160 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.160 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.160 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.161 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getfilelock_latency": { 2023-12-12T10:08:59.161 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.161 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.161 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.161 INFO:teuthology.orchestra.run.smithi042.stdout: "req_link_latency": { 2023-12-12T10:08:59.161 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.161 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.162 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.162 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookup_latency": { 2023-12-12T10:08:59.162 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.162 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.162 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.162 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookuphash_latency": { 2023-12-12T10:08:59.163 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.163 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.163 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.163 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupino_latency": { 2023-12-12T10:08:59.163 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.163 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.163 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.164 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupname_latency": { 2023-12-12T10:08:59.164 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.164 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.164 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.164 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupparent_latency": { 2023-12-12T10:08:59.164 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.164 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.164 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.165 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupsnap_latency": { 2023-12-12T10:08:59.165 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.165 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.165 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.165 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lssnap_latency": { 2023-12-12T10:08:59.165 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.165 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.166 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.166 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mkdir_latency": { 2023-12-12T10:08:59.166 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.166 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.166 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.166 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mknod_latency": { 2023-12-12T10:08:59.167 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.167 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.167 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.167 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mksnap_latency": { 2023-12-12T10:08:59.167 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.167 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.167 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: "req_open_latency": { 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: "req_readdir_latency": { 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.169 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rename_latency": { 2023-12-12T10:08:59.169 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.169 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.169 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.169 INFO:teuthology.orchestra.run.smithi042.stdout: "req_renamesnap_latency": { 2023-12-12T10:08:59.169 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.169 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.170 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.170 INFO:teuthology.orchestra.run.smithi042.stdout: "req_snapdiff_latency": { 2023-12-12T10:08:59.170 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.170 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.170 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.171 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmdir_latency": { 2023-12-12T10:08:59.171 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.171 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.171 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.171 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmsnap_latency": { 2023-12-12T10:08:59.171 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.171 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.171 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.172 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmxattr_latency": { 2023-12-12T10:08:59.172 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.172 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.172 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.172 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setattr_latency": { 2023-12-12T10:08:59.172 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.173 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.173 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.173 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setdirlayout_latency": { 2023-12-12T10:08:59.173 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.173 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.173 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.173 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setfilelock_latency": { 2023-12-12T10:08:59.174 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.174 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.174 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.174 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setlayout_latency": { 2023-12-12T10:08:59.174 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.174 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.174 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.175 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setxattr_latency": { 2023-12-12T10:08:59.175 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.175 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.175 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.175 INFO:teuthology.orchestra.run.smithi042.stdout: "req_symlink_latency": { 2023-12-12T10:08:59.175 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.175 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.175 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.176 INFO:teuthology.orchestra.run.smithi042.stdout: "req_unlink_latency": { 2023-12-12T10:08:59.176 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.176 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.176 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.176 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:08:59.176 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:08:59.176 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getvxattr_latency": { 2023-12-12T10:08:59.177 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.177 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.177 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.177 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:59.177 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.177 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sessions": { 2023-12-12T10:08:59.177 INFO:teuthology.orchestra.run.smithi042.stdout: "session_count": 2, 2023-12-12T10:08:59.177 INFO:teuthology.orchestra.run.smithi042.stdout: "session_add": 3, 2023-12-12T10:08:59.177 INFO:teuthology.orchestra.run.smithi042.stdout: "session_remove": 1, 2023-12-12T10:08:59.178 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_open": 3, 2023-12-12T10:08:59.178 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_stale": 0, 2023-12-12T10:08:59.178 INFO:teuthology.orchestra.run.smithi042.stdout: "total_load": 0, 2023-12-12T10:08:59.178 INFO:teuthology.orchestra.run.smithi042.stdout: "average_load": 0, 2023-12-12T10:08:59.178 INFO:teuthology.orchestra.run.smithi042.stdout: "avg_session_uptime": 2871, 2023-12-12T10:08:59.178 INFO:teuthology.orchestra.run.smithi042.stdout: "mdthresh_evicted": 0 2023-12-12T10:08:59.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.178 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2023-12-12T10:08:59.179 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:08:59.179 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2023-12-12T10:08:59.179 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:08:59.179 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:08:59.179 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:08:59.179 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:08:59.179 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:08:59.179 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:08:59.179 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:08:59.180 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:08:59.180 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:08:59.180 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:08:59.180 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:08:59.180 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:08:59.180 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:08:59.180 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2023-12-12T10:08:59.180 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:08:59.180 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2023-12-12T10:08:59.181 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:08:59.181 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:08:59.181 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:08:59.181 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:08:59.181 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:08:59.181 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:08:59.181 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:08:59.181 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2023-12-12T10:08:59.181 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:08:59.182 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:08:59.182 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:08:59.182 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2023-12-12T10:08:59.182 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2023-12-12T10:08:59.182 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2023-12-12T10:08:59.182 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:08:59.182 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:08:59.182 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:08:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:08:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 231368, 2023-12-12T10:08:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 75, 2023-12-12T10:08:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2023-12-12T10:08:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2023-12-12T10:08:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2023-12-12T10:08:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2023-12-12T10:08:59.183 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:08:59.184 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2023-12-12T10:08:59.184 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:08:59.184 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2023-12-12T10:08:59.184 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 9608, 2023-12-12T10:08:59.184 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 184, 2023-12-12T10:08:59.184 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:08:59.184 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:08:59.184 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2023-12-12T10:08:59.184 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2023-12-12T10:08:59.185 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 7708640, 2023-12-12T10:08:59.185 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 43602, 2023-12-12T10:08:59.185 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2023-12-12T10:08:59.185 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2023-12-12T10:08:59.185 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2023-12-12T10:08:59.185 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2023-12-12T10:08:59.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.186 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": { 2023-12-12T10:08:59.186 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 0, 2023-12-12T10:08:59.186 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2023-12-12T10:08:59.186 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 2223, 2023-12-12T10:08:59.187 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 21, 2023-12-12T10:08:59.187 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 3, 2023-12-12T10:08:59.187 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 2223, 2023-12-12T10:08:59.187 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2023-12-12T10:08:59.188 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 2223, 2023-12-12T10:08:59.188 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 7.632938452, 2023-12-12T10:08:59.188 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.003433620 2023-12-12T10:08:59.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.189 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2023-12-12T10:08:59.189 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2023-12-12T10:08:59.189 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 2223, 2023-12-12T10:08:59.189 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 2224 2023-12-12T10:08:59.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.189 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 2223, 2023-12-12T10:08:59.190 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 2223, 2023-12-12T10:08:59.190 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 0, 2023-12-12T10:08:59.190 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2023-12-12T10:08:59.190 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2023-12-12T10:08:59.190 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 1389, 2023-12-12T10:08:59.190 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 0, 2023-12-12T10:08:59.191 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 832, 2023-12-12T10:08:59.191 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 0, 2023-12-12T10:08:59.191 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 0, 2023-12-12T10:08:59.191 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2023-12-12T10:08:59.191 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2023-12-12T10:08:59.191 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2023-12-12T10:08:59.192 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 0, 2023-12-12T10:08:59.192 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 0, 2023-12-12T10:08:59.192 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2023-12-12T10:08:59.192 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2023-12-12T10:08:59.192 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2023-12-12T10:08:59.193 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 1, 2023-12-12T10:08:59.193 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 0, 2023-12-12T10:08:59.193 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:08:59.193 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2023-12-12T10:08:59.193 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:08:59.193 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2023-12-12T10:08:59.193 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2023-12-12T10:08:59.194 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2023-12-12T10:08:59.194 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:08:59.194 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2023-12-12T10:08:59.194 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:08:59.194 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2023-12-12T10:08:59.194 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2023-12-12T10:08:59.194 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2023-12-12T10:08:59.194 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2023-12-12T10:08:59.195 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2023-12-12T10:08:59.195 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2023-12-12T10:08:59.195 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2023-12-12T10:08:59.195 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2023-12-12T10:08:59.195 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2023-12-12T10:08:59.195 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2023-12-12T10:08:59.195 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2023-12-12T10:08:59.196 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2023-12-12T10:08:59.196 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2023-12-12T10:08:59.196 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2023-12-12T10:08:59.196 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2023-12-12T10:08:59.196 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2023-12-12T10:08:59.196 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2023-12-12T10:08:59.196 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 98, 2023-12-12T10:08:59.196 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2023-12-12T10:08:59.196 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 11, 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 10, 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 10, 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 0, 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 2, 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 0 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: "oft": { 2023-12-12T10:08:59.197 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_objs": 0, 2023-12-12T10:08:59.198 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:08:59.198 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_updates": 0, 2023-12-12T10:08:59.198 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_removes": 0 2023-12-12T10:08:59.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.198 INFO:teuthology.orchestra.run.smithi042.stdout: "osd-slow-ops": { 2023-12-12T10:08:59.198 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_ops_count": 0 2023-12-12T10:08:59.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "purge_queue": { 2023-12-12T10:08:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops": 0, 2023-12-12T10:08:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops_high_water": 0, 2023-12-12T10:08:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing": 0, 2023-12-12T10:08:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_high_water": 0, 2023-12-12T10:08:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executed": 0, 2023-12-12T10:08:59.199 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_item_in_journal": 0 2023-12-12T10:08:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:08:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2023-12-12T10:08:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 2477, 2023-12-12T10:08:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 123527832, 2023-12-12T10:08:59.200 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 2477, 2023-12-12T10:08:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2023-12-12T10:08:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2023-12-12T10:08:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 2477, 2023-12-12T10:08:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 123527832, 2023-12-12T10:08:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:59.201 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": { 2023-12-12T10:08:59.202 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 2223, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 121539853, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 2223, 2023-12-12T10:08:59.203 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 121539853, 2023-12-12T10:08:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": { 2023-12-12T10:08:59.204 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2023-12-12T10:08:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2023-12-12T10:08:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2023-12-12T10:08:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2023-12-12T10:08:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 2223, 2023-12-12T10:08:59.205 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 2223, 2023-12-12T10:08:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 2223, 2023-12-12T10:08:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 2223, 2023-12-12T10:08:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.206 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:08:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2023-12-12T10:08:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2023-12-12T10:08:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2023-12-12T10:08:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2023-12-12T10:08:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2023-12-12T10:08:59.207 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle-0x561abed9e960": { 2023-12-12T10:08:59.208 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 0, 2023-12-12T10:08:59.209 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 0, 2023-12-12T10:08:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:08:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:08:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:08:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:08:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:59.210 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:08:59.210 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T10:08:59.551 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2023-12-12T10:08:59.551 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2023-12-12T10:08:59.551 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2023-12-12T10:08:59.551 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2023-12-12T10:08:59.551 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2023-12-12T10:08:59.552 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2023-12-12T10:08:59.552 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2023-12-12T10:08:59.552 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2023-12-12T10:08:59.552 DEBUG:tasks.check_counter:Getting stats from j 2023-12-12T10:08:59.552 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.j perf dump 2023-12-12T10:09:00.153 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.a/config 2023-12-12T10:09:00.654 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:00 smithi042 ceph-mon[98894]: pgmap v320: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:09:00.747 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:00 smithi111 ceph-mon[102565]: pgmap v320: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:09:00.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:00 smithi173 ceph-mon[103308]: pgmap v320: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:09:01.377 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2023-12-12T10:09:01.378 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:09:01.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 4, 2023-12-12T10:09:01.378 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 4, 2023-12-12T10:09:01.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 2992, 2023-12-12T10:09:01.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 448, 2023-12-12T10:09:01.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2023-12-12T10:09:01.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2023-12-12T10:09:01.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.001722876, 2023-12-12T10:09:01.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000042784, 2023-12-12T10:09:01.379 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000682016, 2023-12-12T10:09:01.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-12T10:09:01.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:01.380 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 3, 2023-12-12T10:09:01.380 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000883440, 2023-12-12T10:09:01.380 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000294480 2023-12-12T10:09:01.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.380 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:01.381 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:09:01.381 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:09:01.381 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:09:01.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.381 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-12T10:09:01.381 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 448 2023-12-12T10:09:01.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.382 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:09:01.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2023-12-12T10:09:01.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 158, 2023-12-12T10:09:01.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2023-12-12T10:09:01.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 181616, 2023-12-12T10:09:01.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 5, 2023-12-12T10:09:01.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 3, 2023-12-12T10:09:01.382 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.012275667, 2023-12-12T10:09:01.383 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.008189428, 2023-12-12T10:09:01.383 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.001737777, 2023-12-12T10:09:01.383 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-12T10:09:01.383 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:01.383 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 155, 2023-12-12T10:09:01.383 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.032777522, 2023-12-12T10:09:01.383 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000211467 2023-12-12T10:09:01.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.384 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:01.384 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:09:01.384 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:09:01.384 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:09:01.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.384 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-12T10:09:01.384 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 181616 2023-12-12T10:09:01.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.385 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:09:01.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 217, 2023-12-12T10:09:01.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 207, 2023-12-12T10:09:01.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 1423376, 2023-12-12T10:09:01.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 124144, 2023-12-12T10:09:01.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2023-12-12T10:09:01.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2023-12-12T10:09:01.385 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.041821600, 2023-12-12T10:09:01.386 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.013064225, 2023-12-12T10:09:01.386 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.032750544, 2023-12-12T10:09:01.386 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.005023001, 2023-12-12T10:09:01.386 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:01.386 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 206, 2023-12-12T10:09:01.386 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.022682120, 2023-12-12T10:09:01.386 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000110107 2023-12-12T10:09:01.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.387 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:01.387 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:09:01.387 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:09:01.387 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:09:01.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.387 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 1423376, 2023-12-12T10:09:01.387 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 124144 2023-12-12T10:09:01.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.388 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2023-12-12T10:09:01.388 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2023-12-12T10:09:01.388 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2023-12-12T10:09:01.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.388 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2023-12-12T10:09:01.388 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:09:01.389 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2023-12-12T10:09:01.389 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:09:01.389 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:09:01.389 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:09:01.389 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:09:01.389 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:09:01.389 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:09:01.390 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:09:01.390 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:09:01.390 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:09:01.390 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:09:01.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:09:01.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:09:01.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:09:01.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2023-12-12T10:09:01.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:09:01.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2023-12-12T10:09:01.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:09:01.391 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:09:01.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:09:01.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:09:01.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:09:01.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:09:01.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:09:01.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2023-12-12T10:09:01.392 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:09:01.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:09:01.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:09:01.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2023-12-12T10:09:01.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2023-12-12T10:09:01.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2023-12-12T10:09:01.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:09:01.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:09:01.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:09:01.393 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:09:01.394 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148984, 2023-12-12T10:09:01.394 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2023-12-12T10:09:01.394 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2023-12-12T10:09:01.394 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2023-12-12T10:09:01.394 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2023-12-12T10:09:01.394 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2023-12-12T10:09:01.394 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:09:01.395 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2023-12-12T10:09:01.395 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:09:01.395 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2023-12-12T10:09:01.395 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2023-12-12T10:09:01.395 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2023-12-12T10:09:01.395 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:09:01.396 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:09:01.396 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2023-12-12T10:09:01.396 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2023-12-12T10:09:01.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2023-12-12T10:09:01.396 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2023-12-12T10:09:01.396 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2023-12-12T10:09:01.396 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2023-12-12T10:09:01.396 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2023-12-12T10:09:01.397 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2023-12-12T10:09:01.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2023-12-12T10:09:01.397 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:09:01.397 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2023-12-12T10:09:01.397 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2023-12-12T10:09:01.397 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2023-12-12T10:09:01.397 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 219, 2023-12-12T10:09:01.397 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 1392787, 2023-12-12T10:09:01.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:01.398 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 219, 2023-12-12T10:09:01.398 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2023-12-12T10:09:01.398 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2023-12-12T10:09:01.398 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 219, 2023-12-12T10:09:01.398 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 1392787, 2023-12-12T10:09:01.398 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2023-12-12T10:09:01.399 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2023-12-12T10:09:01.399 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2023-12-12T10:09:01.399 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2023-12-12T10:09:01.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:09:01.399 INFO:teuthology.orchestra.run.smithi042.stdout: } 2023-12-12T10:09:01.399 INFO:teuthology.orchestra.run.smithi042.stdout:} 2023-12-12T10:09:01.826 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-12-12T10:09:01.826 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-12-12T10:09:01.827 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-12-12T10:09:01.827 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-12-12T10:09:01.827 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-12-12T10:09:01.827 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-12-12T10:09:01.828 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-12-12T10:09:01.828 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-12-12T10:09:01.828 DEBUG:tasks.check_counter:Getting stats from b 2023-12-12T10:09:01.828 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.b perf dump 2023-12-12T10:09:02.445 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T10:09:02.749 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:02 smithi111 ceph-mon[102565]: pgmap v321: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:09:02.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:02 smithi173 ceph-mon[103308]: pgmap v321: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:09:02.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:02 smithi042 ceph-mon[98894]: pgmap v321: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:09:03.667 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-12T10:09:03.667 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:09:03.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 37108, 2023-12-12T10:09:03.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 36239, 2023-12-12T10:09:03.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 16976592, 2023-12-12T10:09:03.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 170706957, 2023-12-12T10:09:03.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 16, 2023-12-12T10:09:03.668 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2023-12-12T10:09:03.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 2.347390761, 2023-12-12T10:09:03.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.760958013, 2023-12-12T10:09:03.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.596140724, 2023-12-12T10:09:03.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 1.018378351, 2023-12-12T10:09:03.669 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:03.669 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 36234, 2023-12-12T10:09:03.669 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 73.443884603, 2023-12-12T10:09:03.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002026932 2023-12-12T10:09:03.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.670 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:03.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 15270, 2023-12-12T10:09:03.670 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002442386, 2023-12-12T10:09:03.670 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000159 2023-12-12T10:09:03.670 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.671 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 98208, 2023-12-12T10:09:03.671 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 342576 2023-12-12T10:09:03.671 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.671 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:09:03.671 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 127903, 2023-12-12T10:09:03.671 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 231212, 2023-12-12T10:09:03.671 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 103959965, 2023-12-12T10:09:03.671 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 197391325, 2023-12-12T10:09:03.672 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 16, 2023-12-12T10:09:03.672 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 8, 2023-12-12T10:09:03.672 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 7.318751654, 2023-12-12T10:09:03.672 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 2.901572915, 2023-12-12T10:09:03.672 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 69.745604543, 2023-12-12T10:09:03.672 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 1.011348834, 2023-12-12T10:09:03.672 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:03.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 231212, 2023-12-12T10:09:03.673 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 129.153680942, 2023-12-12T10:09:03.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000558594 2023-12-12T10:09:03.673 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.673 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:03.673 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 267146, 2023-12-12T10:09:03.673 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.040139342, 2023-12-12T10:09:03.674 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000150 2023-12-12T10:09:03.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.674 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1961776, 2023-12-12T10:09:03.674 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 234992 2023-12-12T10:09:03.674 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.674 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:09:03.674 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 70704, 2023-12-12T10:09:03.674 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 53294, 2023-12-12T10:09:03.675 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 54081631, 2023-12-12T10:09:03.675 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 92100253, 2023-12-12T10:09:03.675 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 63, 2023-12-12T10:09:03.675 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2023-12-12T10:09:03.675 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 3.375904820, 2023-12-12T10:09:03.675 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 1.072509496, 2023-12-12T10:09:03.675 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 1.637455735, 2023-12-12T10:09:03.676 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 1.118310123, 2023-12-12T10:09:03.676 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:03.676 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 53294, 2023-12-12T10:09:03.676 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 85.014337294, 2023-12-12T10:09:03.676 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001595195 2023-12-12T10:09:03.676 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.676 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:03.676 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 60452, 2023-12-12T10:09:03.677 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.007956497, 2023-12-12T10:09:03.677 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000131 2023-12-12T10:09:03.677 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.678 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 42384, 2023-12-12T10:09:03.678 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 126976 2023-12-12T10:09:03.678 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.678 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2023-12-12T10:09:03.678 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2023-12-12T10:09:03.678 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2023-12-12T10:09:03.679 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.679 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2023-12-12T10:09:03.679 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2023-12-12T10:09:03.679 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2023-12-12T10:09:03.679 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 5594, 2023-12-12T10:09:03.679 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 65.639704390, 2023-12-12T10:09:03.679 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.011733947 2023-12-12T10:09:03.679 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:03.680 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.680 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2023-12-12T10:09:03.680 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2023-12-12T10:09:03.680 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2023-12-12T10:09:03.680 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 15463, 2023-12-12T10:09:03.680 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 4.738555802, 2023-12-12T10:09:03.680 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000306444 2023-12-12T10:09:03.680 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:03.680 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.681 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2023-12-12T10:09:03.681 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 63429, 2023-12-12T10:09:03.682 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 52079, 2023-12-12T10:09:03.682 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2023-12-12T10:09:03.682 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 52079, 2023-12-12T10:09:03.682 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2459.418586505, 2023-12-12T10:09:03.682 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.047224765 2023-12-12T10:09:03.682 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.683 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2023-12-12T10:09:03.683 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 11350, 2023-12-12T10:09:03.683 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 651, 2023-12-12T10:09:03.683 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2023-12-12T10:09:03.683 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 8415, 2023-12-12T10:09:03.683 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 208, 2023-12-12T10:09:03.683 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 4, 2023-12-12T10:09:03.683 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 12173, 2023-12-12T10:09:03.684 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 8502, 2023-12-12T10:09:03.684 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 3661, 2023-12-12T10:09:03.684 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 10, 2023-12-12T10:09:03.684 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 27, 2023-12-12T10:09:03.684 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 37542, 2023-12-12T10:09:03.684 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 5, 2023-12-12T10:09:03.684 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 5, 2023-12-12T10:09:03.684 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 6, 2023-12-12T10:09:03.684 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 73105, 2023-12-12T10:09:03.685 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 61410, 2023-12-12T10:09:03.685 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 11194, 2023-12-12T10:09:03.685 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 34, 2023-12-12T10:09:03.685 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 70, 2023-12-12T10:09:03.685 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2023-12-12T10:09:03.685 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2023-12-12T10:09:03.686 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 149, 2023-12-12T10:09:03.686 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2023-12-12T10:09:03.686 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 897, 2023-12-12T10:09:03.686 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 36127, 2023-12-12T10:09:03.686 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 315, 2023-12-12T10:09:03.686 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 13459, 2023-12-12T10:09:03.686 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2023-12-12T10:09:03.686 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:09:03.686 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 246, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 7, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:09:03.687 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 6, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 470, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 42456, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 1, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 4, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 17684, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 17205, 2023-12-12T10:09:03.688 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 139, 2023-12-12T10:09:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 17704 2023-12-12T10:09:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2023-12-12T10:09:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 1779, 2023-12-12T10:09:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 938, 2023-12-12T10:09:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 469, 2023-12-12T10:09:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 1004, 2023-12-12T10:09:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 3771, 2023-12-12T10:09:03.689 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 7, 2023-12-12T10:09:03.690 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2023-12-12T10:09:03.690 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:09:03.690 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 8364, 2023-12-12T10:09:03.690 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 8357, 2023-12-12T10:09:03.690 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2023-12-12T10:09:03.690 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2023-12-12T10:09:03.690 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2023-12-12T10:09:03.690 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:09:03.690 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 932, 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 222, 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.691 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2023-12-12T10:09:03.692 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 4, 2023-12-12T10:09:03.692 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 70360, 2023-12-12T10:09:03.692 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 70289, 2023-12-12T10:09:03.692 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 70289, 2023-12-12T10:09:03.692 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 71, 2023-12-12T10:09:03.692 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2023-12-12T10:09:03.692 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 27, 2023-12-12T10:09:03.692 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 123, 2023-12-12T10:09:03.692 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 122, 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 122, 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 1, 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 1, 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 1, 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 238428601, 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 238649623, 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2023-12-12T10:09:03.693 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2303, 2023-12-12T10:09:03.694 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 188.910470595, 2023-12-12T10:09:03.694 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.082027994 2023-12-12T10:09:03.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.694 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2023-12-12T10:09:03.694 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.694 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2023-12-12T10:09:03.694 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 12151, 2023-12-12T10:09:03.694 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 55251, 2023-12-12T10:09:03.695 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 43100, 2023-12-12T10:09:03.695 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 567, 2023-12-12T10:09:03.695 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 3623, 2023-12-12T10:09:03.695 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 3056, 2023-12-12T10:09:03.695 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 12173, 2023-12-12T10:09:03.695 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 68005, 2023-12-12T10:09:03.696 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 55832, 2023-12-12T10:09:03.696 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 5, 2023-12-12T10:09:03.696 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 39916, 2023-12-12T10:09:03.696 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 39911, 2023-12-12T10:09:03.696 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 251808, 2023-12-12T10:09:03.696 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 207228 2023-12-12T10:09:03.696 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.696 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2023-12-12T10:09:03.696 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 64419, 2023-12-12T10:09:03.697 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 67, 2023-12-12T10:09:03.697 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 63429, 2023-12-12T10:09:03.697 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 693, 2023-12-12T10:09:03.697 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 156, 2023-12-12T10:09:03.697 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2023-12-12T10:09:03.697 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 33573, 2023-12-12T10:09:03.697 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2399.866776683, 2023-12-12T10:09:03.697 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.071482047 2023-12-12T10:09:03.697 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.698 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2023-12-12T10:09:03.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 315, 2023-12-12T10:09:03.698 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.578898717, 2023-12-12T10:09:03.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001837773 2023-12-12T10:09:03.698 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.698 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2023-12-12T10:09:03.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.698 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.698 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.699 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2023-12-12T10:09:03.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.699 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.699 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.699 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2023-12-12T10:09:03.699 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8676, 2023-12-12T10:09:03.700 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 11.947079133, 2023-12-12T10:09:03.700 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001377026 2023-12-12T10:09:03.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.700 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2023-12-12T10:09:03.700 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.700 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.700 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.700 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.700 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2023-12-12T10:09:03.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.701 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.701 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2023-12-12T10:09:03.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.701 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.701 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.701 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.702 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2023-12-12T10:09:03.702 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.702 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.702 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.702 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.702 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2023-12-12T10:09:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2023-12-12T10:09:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 11, 2023-12-12T10:09:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002906703, 2023-12-12T10:09:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000264245 2023-12-12T10:09:03.703 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.704 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2023-12-12T10:09:03.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 125, 2023-12-12T10:09:03.704 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.125062413, 2023-12-12T10:09:03.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001000499 2023-12-12T10:09:03.704 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.704 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2023-12-12T10:09:03.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.704 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.704 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.705 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2023-12-12T10:09:03.705 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 11, 2023-12-12T10:09:03.705 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 3.475522041, 2023-12-12T10:09:03.705 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.315956549 2023-12-12T10:09:03.705 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.705 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2023-12-12T10:09:03.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.706 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.706 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.706 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2023-12-12T10:09:03.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 953, 2023-12-12T10:09:03.706 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 8.814916246, 2023-12-12T10:09:03.706 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.009249649 2023-12-12T10:09:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2023-12-12T10:09:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2, 2023-12-12T10:09:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002734927, 2023-12-12T10:09:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001367463 2023-12-12T10:09:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2023-12-12T10:09:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.708 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.708 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.708 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2023-12-12T10:09:03.708 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.708 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.708 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.708 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.708 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2023-12-12T10:09:03.708 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 112, 2023-12-12T10:09:03.709 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.411426877, 2023-12-12T10:09:03.709 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.003673454 2023-12-12T10:09:03.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.709 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2023-12-12T10:09:03.709 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4, 2023-12-12T10:09:03.709 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.021410848, 2023-12-12T10:09:03.709 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.005352712 2023-12-12T10:09:03.709 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.709 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2023-12-12T10:09:03.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2023-12-12T10:09:03.710 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002153664, 2023-12-12T10:09:03.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000717888 2023-12-12T10:09:03.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.710 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2023-12-12T10:09:03.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4, 2023-12-12T10:09:03.710 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.013332403, 2023-12-12T10:09:03.710 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.003333100 2023-12-12T10:09:03.710 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 22, 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 10.024205641, 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.455645710 2023-12-12T10:09:03.711 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.712 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2023-12-12T10:09:03.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.712 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.712 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.712 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2023-12-12T10:09:03.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 16, 2023-12-12T10:09:03.712 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.032226102, 2023-12-12T10:09:03.712 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002014131 2023-12-12T10:09:03.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.713 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2023-12-12T10:09:03.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.713 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.713 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.713 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2023-12-12T10:09:03.713 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8252, 2023-12-12T10:09:03.713 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 24.099934107, 2023-12-12T10:09:03.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002920496 2023-12-12T10:09:03.714 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.714 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:09:03.714 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:09:03.714 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2023-12-12T10:09:03.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.714 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.714 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.714 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:03.715 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.715 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2023-12-12T10:09:03.715 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 2, 2023-12-12T10:09:03.715 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 6, 2023-12-12T10:09:03.715 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 4, 2023-12-12T10:09:03.715 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 2, 2023-12-12T10:09:03.715 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2023-12-12T10:09:03.715 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 467, 2023-12-12T10:09:03.715 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 233, 2023-12-12T10:09:03.716 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 618, 2023-12-12T10:09:03.716 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2023-12-12T10:09:03.716 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.716 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2023-12-12T10:09:03.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 600, 2023-12-12T10:09:03.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 600, 2023-12-12T10:09:03.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:09:03.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:09:03.716 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:09:03.717 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:09:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2023-12-12T10:09:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:09:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2023-12-12T10:09:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:09:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:09:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:09:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:09:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:09:03.718 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:09:03.719 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:09:03.719 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2023-12-12T10:09:03.719 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:09:03.719 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:09:03.719 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:09:03.719 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2023-12-12T10:09:03.719 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2023-12-12T10:09:03.719 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2023-12-12T10:09:03.719 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 737616, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 11871, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 528, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 6, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2023-12-12T10:09:03.720 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:09:03.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2023-12-12T10:09:03.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:09:03.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2023-12-12T10:09:03.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 9608, 2023-12-12T10:09:03.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 184, 2023-12-12T10:09:03.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:09:03.721 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:09:03.721 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2023-12-12T10:09:03.721 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2023-12-12T10:09:03.722 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 36812894, 2023-12-12T10:09:03.722 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 337058, 2023-12-12T10:09:03.722 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2023-12-12T10:09:03.722 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2023-12-12T10:09:03.722 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2023-12-12T10:09:03.722 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2023-12-12T10:09:03.722 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.722 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2023-12-12T10:09:03.722 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2023-12-12T10:09:03.723 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2023-12-12T10:09:03.723 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 87209, 2023-12-12T10:09:03.723 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 309969336, 2023-12-12T10:09:03.723 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 37, 2023-12-12T10:09:03.723 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 87209, 2023-12-12T10:09:03.723 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2023-12-12T10:09:03.723 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 87209, 2023-12-12T10:09:03.723 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 64501.065483858, 2023-12-12T10:09:03.724 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.739614781 2023-12-12T10:09:03.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.724 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2023-12-12T10:09:03.724 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2023-12-12T10:09:03.724 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 87209, 2023-12-12T10:09:03.724 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 185348 2023-12-12T10:09:03.724 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.724 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 87209, 2023-12-12T10:09:03.724 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 1907, 2023-12-12T10:09:03.725 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 85302, 2023-12-12T10:09:03.725 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2023-12-12T10:09:03.725 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2023-12-12T10:09:03.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 6714, 2023-12-12T10:09:03.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 56654, 2023-12-12T10:09:03.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 173, 2023-12-12T10:09:03.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 2476, 2023-12-12T10:09:03.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 194, 2023-12-12T10:09:03.725 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2023-12-12T10:09:03.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2023-12-12T10:09:03.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2023-12-12T10:09:03.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 5, 2023-12-12T10:09:03.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 17448, 2023-12-12T10:09:03.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2023-12-12T10:09:03.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2023-12-12T10:09:03.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2023-12-12T10:09:03.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2023-12-12T10:09:03.726 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 85517, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:09:03.727 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 1, 2023-12-12T10:09:03.728 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2023-12-12T10:09:03.728 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2023-12-12T10:09:03.728 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2023-12-12T10:09:03.728 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2023-12-12T10:09:03.728 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2023-12-12T10:09:03.728 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2023-12-12T10:09:03.728 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2023-12-12T10:09:03.728 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2023-12-12T10:09:03.728 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 98, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2023-12-12T10:09:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 16, 2023-12-12T10:09:03.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 12, 2023-12-12T10:09:03.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 12, 2023-12-12T10:09:03.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2023-12-12T10:09:03.730 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2023-12-12T10:09:03.730 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 13193, 2023-12-12T10:09:03.730 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2386, 2023-12-12T10:09:03.730 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 587 2023-12-12T10:09:03.730 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.730 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2023-12-12T10:09:03.731 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2023-12-12T10:09:03.732 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 10, 2023-12-12T10:09:03.732 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2023-12-12T10:09:03.732 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 9, 2023-12-12T10:09:03.732 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 8357, 2023-12-12T10:09:03.732 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2023-12-12T10:09:03.732 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.732 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:09:03.732 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:03.732 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2023-12-12T10:09:03.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:03.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 235711, 2023-12-12T10:09:03.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 153737475, 2023-12-12T10:09:03.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:03.733 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 235711, 2023-12-12T10:09:03.733 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:03.733 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:03.733 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 235711, 2023-12-12T10:09:03.733 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 153737475, 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2023-12-12T10:09:03.734 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:03.735 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:03.735 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:03.735 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:03.735 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:03.735 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 87209, 2023-12-12T10:09:03.735 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 310932308, 2023-12-12T10:09:03.735 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 87209, 2023-12-12T10:09:03.735 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 310932308, 2023-12-12T10:09:03.735 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:03.736 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.736 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.736 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.736 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:03.736 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.736 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2023-12-12T10:09:03.736 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:03.736 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2023-12-12T10:09:03.736 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:03.737 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:03.737 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:03.737 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:03.737 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:03.737 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 87209, 2023-12-12T10:09:03.737 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 87209, 2023-12-12T10:09:03.737 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 87209, 2023-12-12T10:09:03.737 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 87209, 2023-12-12T10:09:03.737 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:03.738 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.738 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.738 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.738 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:03.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.738 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:09:03.738 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:03.738 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2023-12-12T10:09:03.738 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:03.739 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 8357, 2023-12-12T10:09:03.739 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1002433, 2023-12-12T10:09:03.739 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:03.739 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 8357, 2023-12-12T10:09:03.739 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:03.739 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:03.739 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 173, 2023-12-12T10:09:03.739 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1002433, 2023-12-12T10:09:03.739 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x556b4ef2a6e0": { 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 60, 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:03.740 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 70360, 2023-12-12T10:09:03.741 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 234455319, 2023-12-12T10:09:03.741 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:03.741 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 70360, 2023-12-12T10:09:03.741 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:03.741 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:03.741 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2303, 2023-12-12T10:09:03.741 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 234455259, 2023-12-12T10:09:03.741 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:03.741 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:03.742 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:03.742 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:03.742 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:03.742 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:03.742 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-12T10:09:03.743 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:03 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:09:03.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:03 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:09:03.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:03 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-12T10:09:03.990 INFO:tasks.check_counter:Daemon mds.b mds.exported=897 2023-12-12T10:09:03.990 INFO:tasks.check_counter:Daemon mds.b mds.imported=315 2023-12-12T10:09:03.990 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1779 2023-12-12T10:09:03.990 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=938 2023-12-12T10:09:03.991 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2023-12-12T10:09:03.991 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2023-12-12T10:09:03.991 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2023-12-12T10:09:03.991 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=208 2023-12-12T10:09:03.991 DEBUG:tasks.check_counter:Getting stats from e 2023-12-12T10:09:03.991 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.e perf dump 2023-12-12T10:09:04.601 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T10:09:04.780 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:04 smithi111 ceph-mon[102565]: pgmap v322: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:09:04.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:04 smithi173 ceph-mon[103308]: pgmap v322: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:09:04.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:04 smithi042 ceph-mon[98894]: pgmap v322: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2023-12-12T10:09:05.808 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-12T10:09:05.808 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:09:05.808 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 1615, 2023-12-12T10:09:05.808 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 1615, 2023-12-12T10:09:05.808 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 105336352, 2023-12-12T10:09:05.809 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 501479, 2023-12-12T10:09:05.809 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 8, 2023-12-12T10:09:05.809 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2023-12-12T10:09:05.809 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.189654788, 2023-12-12T10:09:05.809 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.047856468, 2023-12-12T10:09:05.810 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.082100175, 2023-12-12T10:09:05.810 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.046868439, 2023-12-12T10:09:05.810 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:05.810 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1614, 2023-12-12T10:09:05.810 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.085704707, 2023-12-12T10:09:05.810 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000053100 2023-12-12T10:09:05.810 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.811 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:05.811 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.811 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.811 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.811 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-12T10:09:05.811 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 448 2023-12-12T10:09:05.811 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.811 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:09:05.812 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 494, 2023-12-12T10:09:05.812 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 460, 2023-12-12T10:09:05.812 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 68800609, 2023-12-12T10:09:05.812 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 200534, 2023-12-12T10:09:05.812 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 6, 2023-12-12T10:09:05.812 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 5, 2023-12-12T10:09:05.812 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.099053787, 2023-12-12T10:09:05.812 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.016538624, 2023-12-12T10:09:05.813 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.078451259, 2023-12-12T10:09:05.813 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.011878667, 2023-12-12T10:09:05.813 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:05.813 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 459, 2023-12-12T10:09:05.813 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.035434309, 2023-12-12T10:09:05.813 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000077198 2023-12-12T10:09:05.813 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.813 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:05.814 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.814 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.814 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.814 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1600928, 2023-12-12T10:09:05.814 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 127072 2023-12-12T10:09:05.814 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.814 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:09:05.815 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 747, 2023-12-12T10:09:05.815 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 902, 2023-12-12T10:09:05.815 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 62680825, 2023-12-12T10:09:05.815 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 655913, 2023-12-12T10:09:05.815 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 8, 2023-12-12T10:09:05.815 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2023-12-12T10:09:05.816 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.107188230, 2023-12-12T10:09:05.816 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.021057800, 2023-12-12T10:09:05.816 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.055445706, 2023-12-12T10:09:05.816 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.023043590, 2023-12-12T10:09:05.816 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:05.816 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 899, 2023-12-12T10:09:05.817 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.057125817, 2023-12-12T10:09:05.817 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000063543 2023-12-12T10:09:05.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.817 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:05.817 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.817 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.817 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.817 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.817 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-12T10:09:05.818 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 424528 2023-12-12T10:09:05.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.818 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2023-12-12T10:09:05.818 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2023-12-12T10:09:05.818 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2023-12-12T10:09:05.818 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.819 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2023-12-12T10:09:05.819 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2023-12-12T10:09:05.819 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2023-12-12T10:09:05.819 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4167, 2023-12-12T10:09:05.819 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.596880512, 2023-12-12T10:09:05.819 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000143239 2023-12-12T10:09:05.819 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:05.819 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.820 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2023-12-12T10:09:05.820 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2023-12-12T10:09:05.820 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2023-12-12T10:09:05.820 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.820 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.820 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.820 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:05.821 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.821 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2023-12-12T10:09:05.821 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2023-12-12T10:09:05.821 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2023-12-12T10:09:05.821 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2023-12-12T10:09:05.821 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.822 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.822 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.822 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.822 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2023-12-12T10:09:05.822 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2023-12-12T10:09:05.822 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2023-12-12T10:09:05.823 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2023-12-12T10:09:05.823 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 0, 2023-12-12T10:09:05.823 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2023-12-12T10:09:05.823 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2023-12-12T10:09:05.824 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 28685, 2023-12-12T10:09:05.824 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 20058, 2023-12-12T10:09:05.824 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 8627, 2023-12-12T10:09:05.824 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2023-12-12T10:09:05.824 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 31, 2023-12-12T10:09:05.825 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2023-12-12T10:09:05.825 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2023-12-12T10:09:05.825 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2023-12-12T10:09:05.825 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 2, 2023-12-12T10:09:05.826 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2023-12-12T10:09:05.826 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2023-12-12T10:09:05.826 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2023-12-12T10:09:05.826 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2023-12-12T10:09:05.827 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2023-12-12T10:09:05.827 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2023-12-12T10:09:05.827 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2023-12-12T10:09:05.827 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2023-12-12T10:09:05.827 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2023-12-12T10:09:05.827 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2023-12-12T10:09:05.828 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2023-12-12T10:09:05.828 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2023-12-12T10:09:05.828 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2023-12-12T10:09:05.828 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2023-12-12T10:09:05.828 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:09:05.829 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2023-12-12T10:09:05.829 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2023-12-12T10:09:05.829 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 246, 2023-12-12T10:09:05.829 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 7, 2023-12-12T10:09:05.829 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:09:05.830 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2023-12-12T10:09:05.830 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:09:05.830 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:09:05.830 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:09:05.831 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:09:05.831 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:09:05.831 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2023-12-12T10:09:05.831 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2023-12-12T10:09:05.832 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2023-12-12T10:09:05.832 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2023-12-12T10:09:05.832 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2023-12-12T10:09:05.832 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-12T10:09:05.833 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:09:05.833 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2023-12-12T10:09:05.833 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2023-12-12T10:09:05.833 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2023-12-12T10:09:05.833 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2023-12-12T10:09:05.834 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.834 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2023-12-12T10:09:05.834 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2023-12-12T10:09:05.834 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2023-12-12T10:09:05.835 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2023-12-12T10:09:05.835 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2023-12-12T10:09:05.835 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2023-12-12T10:09:05.835 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2023-12-12T10:09:05.836 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2023-12-12T10:09:05.836 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:09:05.836 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2023-12-12T10:09:05.836 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2023-12-12T10:09:05.837 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2023-12-12T10:09:05.837 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2023-12-12T10:09:05.837 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2023-12-12T10:09:05.837 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:09:05.837 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:09:05.838 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2023-12-12T10:09:05.838 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2023-12-12T10:09:05.838 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:09:05.838 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2023-12-12T10:09:05.838 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2023-12-12T10:09:05.838 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2023-12-12T10:09:05.838 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2023-12-12T10:09:05.839 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2023-12-12T10:09:05.839 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.839 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2023-12-12T10:09:05.839 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2023-12-12T10:09:05.839 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 0, 2023-12-12T10:09:05.839 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2023-12-12T10:09:05.839 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2023-12-12T10:09:05.839 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 70359, 2023-12-12T10:09:05.839 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2023-12-12T10:09:05.840 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2023-12-12T10:09:05.840 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 0, 2023-12-12T10:09:05.840 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2023-12-12T10:09:05.840 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2023-12-12T10:09:05.840 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 2, 2023-12-12T10:09:05.840 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 29, 2023-12-12T10:09:05.840 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2023-12-12T10:09:05.841 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2023-12-12T10:09:05.841 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 238428601, 2023-12-12T10:09:05.841 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 238649563, 2023-12-12T10:09:05.841 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 238649479, 2023-12-12T10:09:05.841 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2023-12-12T10:09:05.841 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.841 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.841 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.841 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.842 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 70359 2023-12-12T10:09:05.842 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.842 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2023-12-12T10:09:05.842 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 11428, 2023-12-12T10:09:05.842 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 52122, 2023-12-12T10:09:05.842 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 40694, 2023-12-12T10:09:05.842 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 536, 2023-12-12T10:09:05.842 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 3096, 2023-12-12T10:09:05.843 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 2560, 2023-12-12T10:09:05.843 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 28685, 2023-12-12T10:09:05.843 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 65977, 2023-12-12T10:09:05.843 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 37292, 2023-12-12T10:09:05.843 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2023-12-12T10:09:05.843 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2023-12-12T10:09:05.843 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2023-12-12T10:09:05.843 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 135796, 2023-12-12T10:09:05.843 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 207228 2023-12-12T10:09:05.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.844 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2023-12-12T10:09:05.844 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2023-12-12T10:09:05.844 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2023-12-12T10:09:05.844 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2023-12-12T10:09:05.844 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 0, 2023-12-12T10:09:05.844 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2023-12-12T10:09:05.844 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2023-12-12T10:09:05.844 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.845 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.845 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.845 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2023-12-12T10:09:05.845 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.845 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.845 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.845 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.845 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2023-12-12T10:09:05.846 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.846 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.846 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.846 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.846 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2023-12-12T10:09:05.846 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.846 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.846 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2023-12-12T10:09:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2023-12-12T10:09:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.847 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.848 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.848 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2023-12-12T10:09:05.848 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.848 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.848 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.848 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.848 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2023-12-12T10:09:05.849 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.849 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.849 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.849 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.849 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2023-12-12T10:09:05.849 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.849 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.850 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.850 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.850 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2023-12-12T10:09:05.850 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.851 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.851 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.851 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.851 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2023-12-12T10:09:05.852 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.852 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.852 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.852 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.852 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2023-12-12T10:09:05.853 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.853 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.853 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.854 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.854 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2023-12-12T10:09:05.854 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.854 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.855 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.855 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.855 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2023-12-12T10:09:05.855 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.856 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.856 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.856 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.857 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2023-12-12T10:09:05.857 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.857 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.857 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.858 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.858 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2023-12-12T10:09:05.858 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.859 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.859 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.859 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.860 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2023-12-12T10:09:05.860 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.860 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.861 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.861 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.861 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2023-12-12T10:09:05.861 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.862 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.862 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.862 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.862 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2023-12-12T10:09:05.863 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.863 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.863 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.864 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.864 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2023-12-12T10:09:05.864 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.864 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.864 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.865 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.865 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2023-12-12T10:09:05.865 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.865 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.866 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.866 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.866 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2023-12-12T10:09:05.867 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.867 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.867 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.867 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.868 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2023-12-12T10:09:05.868 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.868 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.868 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.869 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.869 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2023-12-12T10:09:05.869 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.869 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.870 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.870 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.870 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2023-12-12T10:09:05.870 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.871 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.871 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.871 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.871 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2023-12-12T10:09:05.871 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.872 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.872 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.872 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.872 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2023-12-12T10:09:05.873 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.873 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.873 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.873 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.874 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2023-12-12T10:09:05.874 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.874 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.874 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.875 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.875 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2023-12-12T10:09:05.875 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.875 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.876 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.876 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.876 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:09:05.876 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:09:05.877 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2023-12-12T10:09:05.877 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.877 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.877 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.878 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:05.878 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.878 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2023-12-12T10:09:05.878 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 2, 2023-12-12T10:09:05.879 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 6, 2023-12-12T10:09:05.879 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 4, 2023-12-12T10:09:05.879 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 4, 2023-12-12T10:09:05.879 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2023-12-12T10:09:05.880 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2023-12-12T10:09:05.880 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2023-12-12T10:09:05.880 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 6250, 2023-12-12T10:09:05.880 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2023-12-12T10:09:05.881 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.881 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2023-12-12T10:09:05.881 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:09:05.881 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2023-12-12T10:09:05.881 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:09:05.881 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:09:05.882 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:09:05.882 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:09:05.882 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:09:05.882 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:09:05.883 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:09:05.883 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:09:05.883 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:09:05.883 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:09:05.883 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:09:05.883 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:09:05.884 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:09:05.884 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2023-12-12T10:09:05.884 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:09:05.884 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2023-12-12T10:09:05.884 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:09:05.885 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:09:05.885 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:09:05.885 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:09:05.885 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:09:05.885 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:09:05.886 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:09:05.886 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2023-12-12T10:09:05.886 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:09:05.886 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:09:05.886 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:09:05.886 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2023-12-12T10:09:05.886 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2023-12-12T10:09:05.886 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2023-12-12T10:09:05.887 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:09:05.887 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:09:05.887 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:09:05.887 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:09:05.887 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 627314, 2023-12-12T10:09:05.887 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 11507, 2023-12-12T10:09:05.887 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2023-12-12T10:09:05.887 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2023-12-12T10:09:05.887 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2023-12-12T10:09:05.888 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2023-12-12T10:09:05.888 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:09:05.888 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2023-12-12T10:09:05.888 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:09:05.888 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2023-12-12T10:09:05.888 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 9608, 2023-12-12T10:09:05.888 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 184, 2023-12-12T10:09:05.889 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:09:05.889 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:09:05.889 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2023-12-12T10:09:05.889 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2023-12-12T10:09:05.889 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 46592078, 2023-12-12T10:09:05.889 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 367445, 2023-12-12T10:09:05.890 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2023-12-12T10:09:05.890 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2023-12-12T10:09:05.890 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2023-12-12T10:09:05.890 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2023-12-12T10:09:05.891 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.891 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2023-12-12T10:09:05.891 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2023-12-12T10:09:05.891 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2023-12-12T10:09:05.891 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 2591, 2023-12-12T10:09:05.892 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 26, 2023-12-12T10:09:05.892 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 4, 2023-12-12T10:09:05.892 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 2591, 2023-12-12T10:09:05.892 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2023-12-12T10:09:05.892 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2591, 2023-12-12T10:09:05.893 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 6.425885126, 2023-12-12T10:09:05.893 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002480079 2023-12-12T10:09:05.893 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.893 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2023-12-12T10:09:05.894 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2023-12-12T10:09:05.894 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2591, 2023-12-12T10:09:05.894 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2592 2023-12-12T10:09:05.894 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.895 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 2591, 2023-12-12T10:09:05.895 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 2591, 2023-12-12T10:09:05.895 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 0, 2023-12-12T10:09:05.895 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2023-12-12T10:09:05.896 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2023-12-12T10:09:05.896 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 1623, 2023-12-12T10:09:05.896 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 0, 2023-12-12T10:09:05.896 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 965, 2023-12-12T10:09:05.896 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 0, 2023-12-12T10:09:05.896 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 0, 2023-12-12T10:09:05.897 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2023-12-12T10:09:05.897 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2023-12-12T10:09:05.897 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2023-12-12T10:09:05.897 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2023-12-12T10:09:05.898 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 0, 2023-12-12T10:09:05.898 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2023-12-12T10:09:05.898 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2023-12-12T10:09:05.898 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2023-12-12T10:09:05.899 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 2, 2023-12-12T10:09:05.899 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 0, 2023-12-12T10:09:05.899 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:09:05.899 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2023-12-12T10:09:05.900 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:09:05.900 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2023-12-12T10:09:05.900 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2023-12-12T10:09:05.900 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2023-12-12T10:09:05.900 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:09:05.901 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2023-12-12T10:09:05.901 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:09:05.901 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2023-12-12T10:09:05.901 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2023-12-12T10:09:05.901 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2023-12-12T10:09:05.901 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2023-12-12T10:09:05.901 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2023-12-12T10:09:05.901 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2023-12-12T10:09:05.901 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2023-12-12T10:09:05.902 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2023-12-12T10:09:05.902 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2023-12-12T10:09:05.902 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2023-12-12T10:09:05.902 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2023-12-12T10:09:05.902 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2023-12-12T10:09:05.902 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2023-12-12T10:09:05.902 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2023-12-12T10:09:05.902 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2023-12-12T10:09:05.903 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2023-12-12T10:09:05.903 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2023-12-12T10:09:05.903 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 98, 2023-12-12T10:09:05.903 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2023-12-12T10:09:05.903 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 11, 2023-12-12T10:09:05.903 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 11, 2023-12-12T10:09:05.903 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 11, 2023-12-12T10:09:05.903 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2023-12-12T10:09:05.904 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2023-12-12T10:09:05.904 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 0, 2023-12-12T10:09:05.904 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2, 2023-12-12T10:09:05.904 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 0 2023-12-12T10:09:05.904 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.904 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2023-12-12T10:09:05.904 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 0, 2023-12-12T10:09:05.904 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:09:05.904 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2023-12-12T10:09:05.905 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2023-12-12T10:09:05.905 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.905 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2023-12-12T10:09:05.905 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2023-12-12T10:09:05.905 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.905 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2023-12-12T10:09:05.905 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2023-12-12T10:09:05.905 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2023-12-12T10:09:05.905 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2023-12-12T10:09:05.906 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2023-12-12T10:09:05.906 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2023-12-12T10:09:05.906 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2023-12-12T10:09:05.906 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.906 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:09:05.906 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:05.906 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2023-12-12T10:09:05.906 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:05.907 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 2852, 2023-12-12T10:09:05.907 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 236545033, 2023-12-12T10:09:05.907 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:05.907 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 2852, 2023-12-12T10:09:05.907 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:05.907 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:05.907 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2852, 2023-12-12T10:09:05.907 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 236545033, 2023-12-12T10:09:05.907 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:05.908 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.908 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.908 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.908 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:05.908 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.908 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2023-12-12T10:09:05.908 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:05.908 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2023-12-12T10:09:05.908 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:05.909 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:05.909 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:05.909 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:05.909 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:05.909 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2591, 2023-12-12T10:09:05.909 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 234494946, 2023-12-12T10:09:05.909 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2591, 2023-12-12T10:09:05.909 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 234494946, 2023-12-12T10:09:05.909 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:05.910 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.910 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.910 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.910 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:05.911 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.911 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2023-12-12T10:09:05.911 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:05.911 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2023-12-12T10:09:05.911 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:05.912 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:05.912 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:05.912 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:05.912 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:05.912 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2591, 2023-12-12T10:09:05.913 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 2591, 2023-12-12T10:09:05.913 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2591, 2023-12-12T10:09:05.913 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2591, 2023-12-12T10:09:05.913 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:05.914 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.914 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.914 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.914 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:05.914 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.914 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:09:05.914 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:05.914 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2023-12-12T10:09:05.914 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:05.915 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:05.915 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:05.915 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:05.915 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:05.915 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:05.915 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:05.915 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2023-12-12T10:09:05.915 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2023-12-12T10:09:05.915 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:05.916 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.916 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.916 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.916 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:05.916 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:05.916 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x55cc6bc04960": { 2023-12-12T10:09:05.916 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:05.916 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2023-12-12T10:09:05.916 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:05.917 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:05.917 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:05.917 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:05.917 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:05.917 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:05.917 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:05.917 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2023-12-12T10:09:05.917 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2023-12-12T10:09:05.917 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:05.918 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:05.918 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:05.918 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:05.918 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:05.918 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:05.918 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-12T10:09:06.165 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2023-12-12T10:09:06.166 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2023-12-12T10:09:06.166 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2023-12-12T10:09:06.166 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2023-12-12T10:09:06.166 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=7 2023-12-12T10:09:06.166 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-12-12T10:09:06.166 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-12-12T10:09:06.166 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2023-12-12T10:09:06.166 DEBUG:tasks.check_counter:Getting stats from h 2023-12-12T10:09:06.167 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.h perf dump 2023-12-12T10:09:06.771 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T10:09:07.145 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: pgmap v323: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 25 op/s 2023-12-12T10:09:07.145 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.145 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.145 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.146 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.146 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.146 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.146 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.146 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.146 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.146 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.146 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.147 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.147 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.147 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.147 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.147 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.147 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.147 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.147 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:09:07.148 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:09:07.148 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:06 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: pgmap v323: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 25 op/s 2023-12-12T10:09:07.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:09:07.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:09:07.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:06 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: pgmap v323: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 46 KiB/s wr, 25 op/s 2023-12-12T10:09:07.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:07.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-12T10:09:07.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-12T10:09:07.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-12T10:09:07.326 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:06 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' 2023-12-12T10:09:08.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:07 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi042 to 197.2M 2023-12-12T10:09:08.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:07 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi111 to 196.2M 2023-12-12T10:09:08.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:07 smithi111 ceph-mon[102565]: Unable to set osd_memory_target on smithi111 to 205768192: error parsing value: Value '205768192' is below minimum 939524096 2023-12-12T10:09:08.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:07 smithi111 ceph-mon[102565]: Unable to set osd_memory_target on smithi042 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-12-12T10:09:08.002 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:07 smithi111 ceph-mon[102565]: Adjusting osd_memory_target on smithi173 to 196.2M 2023-12-12T10:09:08.002 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:07 smithi111 ceph-mon[102565]: Unable to set osd_memory_target on smithi173 to 205771776: error parsing value: Value '205771776' is below minimum 939524096 2023-12-12T10:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-12T10:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:09:08.070 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 82029, 2023-12-12T10:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 125388, 2023-12-12T10:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 89830565, 2023-12-12T10:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 117562618, 2023-12-12T10:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 10, 2023-12-12T10:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 3, 2023-12-12T10:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 4.654554245, 2023-12-12T10:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 1.947731544, 2023-12-12T10:09:08.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 147.638710175, 2023-12-12T10:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.303109959, 2023-12-12T10:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 125390, 2023-12-12T10:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 21.483681832, 2023-12-12T10:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000171334 2023-12-12T10:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 171209, 2023-12-12T10:09:08.072 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.030508374, 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000178 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 21243, 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 21171, 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 6706168, 2023-12-12T10:09:08.073 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 74860245, 2023-12-12T10:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 13, 2023-12-12T10:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 9, 2023-12-12T10:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 1.499365074, 2023-12-12T10:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.536711541, 2023-12-12T10:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.527957753, 2023-12-12T10:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.476016183, 2023-12-12T10:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:08.074 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 21169, 2023-12-12T10:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2.824320146, 2023-12-12T10:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000133417 2023-12-12T10:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:08.075 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 16013, 2023-12-12T10:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002767889, 2023-12-12T10:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000172 2023-12-12T10:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 4240, 2023-12-12T10:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 427600 2023-12-12T10:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 34768, 2023-12-12T10:09:08.076 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 34175, 2023-12-12T10:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 75289864, 2023-12-12T10:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 79900147, 2023-12-12T10:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 16, 2023-12-12T10:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 10, 2023-12-12T10:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 1.975962297, 2023-12-12T10:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.688363040, 2023-12-12T10:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 1.231256299, 2023-12-12T10:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.366160104, 2023-12-12T10:09:08.077 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 34170, 2023-12-12T10:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 4.085014610, 2023-12-12T10:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000119549 2023-12-12T10:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 46578, 2023-12-12T10:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.007409722, 2023-12-12T10:09:08.078 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000159 2023-12-12T10:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1622608, 2023-12-12T10:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 140352 2023-12-12T10:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2023-12-12T10:09:08.079 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2023-12-12T10:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2023-12-12T10:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2023-12-12T10:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2023-12-12T10:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2023-12-12T10:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 7108, 2023-12-12T10:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 35.083756701, 2023-12-12T10:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.004935812 2023-12-12T10:09:08.080 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2023-12-12T10:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2023-12-12T10:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2023-12-12T10:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 18973, 2023-12-12T10:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 5.376194998, 2023-12-12T10:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000283360 2023-12-12T10:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:08.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2023-12-12T10:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 36905, 2023-12-12T10:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 29301, 2023-12-12T10:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2023-12-12T10:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 29301, 2023-12-12T10:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 78.742900246, 2023-12-12T10:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.002687379 2023-12-12T10:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.082 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2023-12-12T10:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 7604, 2023-12-12T10:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 330, 2023-12-12T10:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2023-12-12T10:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 556, 2023-12-12T10:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 68, 2023-12-12T10:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 1, 2023-12-12T10:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 21951, 2023-12-12T10:09:08.083 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 3, 2023-12-12T10:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 7, 2023-12-12T10:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 21941, 2023-12-12T10:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 21946, 2023-12-12T10:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 81694, 2023-12-12T10:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2023-12-12T10:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2023-12-12T10:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 5, 2023-12-12T10:09:08.084 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 49548, 2023-12-12T10:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 41117, 2023-12-12T10:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 7481, 2023-12-12T10:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 299, 2023-12-12T10:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 330, 2023-12-12T10:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2023-12-12T10:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2023-12-12T10:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2023-12-12T10:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2023-12-12T10:09:08.085 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 1960, 2023-12-12T10:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 80756, 2023-12-12T10:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 1784, 2023-12-12T10:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 80163, 2023-12-12T10:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2023-12-12T10:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2023-12-12T10:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 4, 2023-12-12T10:09:08.086 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 246, 2023-12-12T10:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 6, 2023-12-12T10:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2023-12-12T10:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:09:08.087 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2023-12-12T10:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2023-12-12T10:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 17, 2023-12-12T10:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 19648, 2023-12-12T10:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2023-12-12T10:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-12T10:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 19436, 2023-12-12T10:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 19418, 2023-12-12T10:09:08.088 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 185, 2023-12-12T10:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 21441 2023-12-12T10:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2023-12-12T10:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 1616, 2023-12-12T10:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 3873, 2023-12-12T10:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 1938, 2023-12-12T10:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 3186, 2023-12-12T10:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 2250, 2023-12-12T10:09:08.089 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 1, 2023-12-12T10:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2023-12-12T10:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 10370, 2023-12-12T10:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 10369, 2023-12-12T10:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2023-12-12T10:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2023-12-12T10:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2023-12-12T10:09:08.090 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2023-12-12T10:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2023-12-12T10:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 2215, 2023-12-12T10:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2023-12-12T10:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 69, 2023-12-12T10:09:08.091 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2023-12-12T10:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2023-12-12T10:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2023-12-12T10:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 3, 2023-12-12T10:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 65321, 2023-12-12T10:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2023-12-12T10:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2023-12-12T10:09:08.092 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 65321, 2023-12-12T10:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2023-12-12T10:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2023-12-12T10:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 91, 2023-12-12T10:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2023-12-12T10:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2023-12-12T10:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 91, 2023-12-12T10:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 7, 2023-12-12T10:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2023-12-12T10:09:08.093 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2023-12-12T10:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2023-12-12T10:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 179188894, 2023-12-12T10:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 0, 2023-12-12T10:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2023-12-12T10:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 4458, 2023-12-12T10:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 185.637870163, 2023-12-12T10:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.041641514 2023-12-12T10:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.094 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 0 2023-12-12T10:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2023-12-12T10:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 11958, 2023-12-12T10:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 70495, 2023-12-12T10:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 58537, 2023-12-12T10:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 536, 2023-12-12T10:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 3296, 2023-12-12T10:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 2760, 2023-12-12T10:09:08.095 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 21951, 2023-12-12T10:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 114014, 2023-12-12T10:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 92063, 2023-12-12T10:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2023-12-12T10:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 21034, 2023-12-12T10:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 21034, 2023-12-12T10:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 210112, 2023-12-12T10:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 199036 2023-12-12T10:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.096 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2023-12-12T10:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 38023, 2023-12-12T10:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 3, 2023-12-12T10:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 36905, 2023-12-12T10:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 251, 2023-12-12T10:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 466, 2023-12-12T10:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2023-12-12T10:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 8538, 2023-12-12T10:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 13.931117463, 2023-12-12T10:09:08.097 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001631660 2023-12-12T10:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2023-12-12T10:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3, 2023-12-12T10:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.002179848, 2023-12-12T10:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000726616 2023-12-12T10:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2023-12-12T10:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.098 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2023-12-12T10:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2023-12-12T10:09:08.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 9755, 2023-12-12T10:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 18.146905144, 2023-12-12T10:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001860267 2023-12-12T10:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2023-12-12T10:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2023-12-12T10:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2023-12-12T10:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2023-12-12T10:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2023-12-12T10:09:08.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2023-12-12T10:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2023-12-12T10:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2023-12-12T10:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2023-12-12T10:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2023-12-12T10:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.105 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2023-12-12T10:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 635, 2023-12-12T10:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 6.212045967, 2023-12-12T10:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.009782749 2023-12-12T10:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2023-12-12T10:09:08.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2023-12-12T10:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.107 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2023-12-12T10:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2023-12-12T10:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1, 2023-12-12T10:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.029647659, 2023-12-12T10:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.029647659 2023-12-12T10:09:08.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2023-12-12T10:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2023-12-12T10:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.109 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2023-12-12T10:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2023-12-12T10:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.110 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2023-12-12T10:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2023-12-12T10:09:08.111 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2023-12-12T10:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.112 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2023-12-12T10:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2023-12-12T10:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 10369, 2023-12-12T10:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 40.421004165, 2023-12-12T10:09:08.113 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.003898254 2023-12-12T10:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2023-12-12T10:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.114 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2023-12-12T10:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 2, 2023-12-12T10:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 4, 2023-12-12T10:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 2, 2023-12-12T10:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 2, 2023-12-12T10:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2023-12-12T10:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 8066, 2023-12-12T10:09:08.115 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 4033, 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 578, 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:09:08.116 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:09:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2023-12-12T10:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2023-12-12T10:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:09:08.118 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2023-12-12T10:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:09:08.119 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2023-12-12T10:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2023-12-12T10:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2023-12-12T10:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 693579, 2023-12-12T10:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 12041, 2023-12-12T10:09:08.120 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 9848, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 190, 2023-12-12T10:09:08.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:09:08.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:09:08.122 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2023-12-12T10:09:08.122 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2023-12-12T10:09:08.122 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 43362905, 2023-12-12T10:09:08.122 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 393347, 2023-12-12T10:09:08.122 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2023-12-12T10:09:08.123 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2023-12-12T10:09:08.123 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2023-12-12T10:09:08.123 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2023-12-12T10:09:08.123 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.123 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2023-12-12T10:09:08.123 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2023-12-12T10:09:08.123 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2023-12-12T10:09:08.123 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 27098, 2023-12-12T10:09:08.124 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 182679215, 2023-12-12T10:09:08.124 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 27, 2023-12-12T10:09:08.124 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 27098, 2023-12-12T10:09:08.124 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2023-12-12T10:09:08.124 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 27098, 2023-12-12T10:09:08.124 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 261.056324188, 2023-12-12T10:09:08.124 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.009633785 2023-12-12T10:09:08.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.125 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2023-12-12T10:09:08.125 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2023-12-12T10:09:08.125 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 27098, 2023-12-12T10:09:08.125 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 28047 2023-12-12T10:09:08.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.125 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 27098, 2023-12-12T10:09:08.125 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 893, 2023-12-12T10:09:08.125 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 26205, 2023-12-12T10:09:08.126 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2023-12-12T10:09:08.126 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2023-12-12T10:09:08.126 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 0, 2023-12-12T10:09:08.126 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 10, 2023-12-12T10:09:08.126 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 185, 2023-12-12T10:09:08.126 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 4643, 2023-12-12T10:09:08.126 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 89, 2023-12-12T10:09:08.126 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 1, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 20866, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 0, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 20, 2023-12-12T10:09:08.127 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:09:08.128 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2023-12-12T10:09:08.128 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:09:08.128 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2023-12-12T10:09:08.128 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2023-12-12T10:09:08.128 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2023-12-12T10:09:08.128 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:09:08.128 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2023-12-12T10:09:08.128 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:09:08.128 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2023-12-12T10:09:08.129 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2023-12-12T10:09:08.129 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2023-12-12T10:09:08.129 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2023-12-12T10:09:08.129 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2023-12-12T10:09:08.129 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2023-12-12T10:09:08.129 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2023-12-12T10:09:08.129 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2023-12-12T10:09:08.129 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2023-12-12T10:09:08.129 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2023-12-12T10:09:08.130 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2023-12-12T10:09:08.130 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2023-12-12T10:09:08.130 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2023-12-12T10:09:08.130 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2023-12-12T10:09:08.130 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2023-12-12T10:09:08.130 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2023-12-12T10:09:08.130 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2023-12-12T10:09:08.130 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 99, 2023-12-12T10:09:08.130 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 12, 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 12, 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 12, 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 1165, 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 1038, 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 30 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.131 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2023-12-12T10:09:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 1, 2023-12-12T10:09:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:09:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2023-12-12T10:09:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2023-12-12T10:09:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2023-12-12T10:09:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2023-12-12T10:09:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.132 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2023-12-12T10:09:08.133 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2023-12-12T10:09:08.133 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 10, 2023-12-12T10:09:08.133 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2023-12-12T10:09:08.133 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 5, 2023-12-12T10:09:08.133 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 10369, 2023-12-12T10:09:08.133 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2023-12-12T10:09:08.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.133 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:09:08.133 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:08.134 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2023-12-12T10:09:08.134 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:08.134 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 138036, 2023-12-12T10:09:08.134 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 159385204, 2023-12-12T10:09:08.134 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:08.134 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 138036, 2023-12-12T10:09:08.135 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:08.135 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:08.135 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 138036, 2023-12-12T10:09:08.135 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 159385204, 2023-12-12T10:09:08.135 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:08.135 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.135 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.135 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.135 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:08.136 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.136 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2023-12-12T10:09:08.137 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:08.137 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2023-12-12T10:09:08.137 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:08.137 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:08.137 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:08.137 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:08.137 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:08.138 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 27098, 2023-12-12T10:09:08.138 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 183924424, 2023-12-12T10:09:08.138 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 27098, 2023-12-12T10:09:08.138 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 183924424, 2023-12-12T10:09:08.138 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:08.138 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.138 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.138 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.138 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:08.139 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.139 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2023-12-12T10:09:08.139 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:08.139 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2023-12-12T10:09:08.139 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:08.139 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:08.139 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:08.139 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 27098, 2023-12-12T10:09:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 27098, 2023-12-12T10:09:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 27098, 2023-12-12T10:09:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 27098, 2023-12-12T10:09:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.141 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:08.141 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.141 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:09:08.141 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:08.141 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2023-12-12T10:09:08.141 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:08.141 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 10369, 2023-12-12T10:09:08.141 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 1261373, 2023-12-12T10:09:08.141 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:08.142 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 10369, 2023-12-12T10:09:08.142 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:08.142 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:08.142 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 185, 2023-12-12T10:09:08.142 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 1261373, 2023-12-12T10:09:08.142 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:08.142 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.142 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.142 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x55a74d4686e0": { 2023-12-12T10:09:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2023-12-12T10:09:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 65321, 2023-12-12T10:09:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 174994590, 2023-12-12T10:09:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:08.144 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 65321, 2023-12-12T10:09:08.144 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:08.144 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:08.144 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 4458, 2023-12-12T10:09:08.144 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 174994590, 2023-12-12T10:09:08.144 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:08.144 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:08.144 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:08.144 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:08.145 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:08.145 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:08.145 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-12T10:09:08.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:07 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi042 to 197.2M 2023-12-12T10:09:08.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:07 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi111 to 196.2M 2023-12-12T10:09:08.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:07 smithi042 ceph-mon[98894]: Unable to set osd_memory_target on smithi111 to 205768192: error parsing value: Value '205768192' is below minimum 939524096 2023-12-12T10:09:08.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:07 smithi042 ceph-mon[98894]: Unable to set osd_memory_target on smithi042 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-12-12T10:09:08.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:07 smithi042 ceph-mon[98894]: Adjusting osd_memory_target on smithi173 to 196.2M 2023-12-12T10:09:08.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:07 smithi042 ceph-mon[98894]: Unable to set osd_memory_target on smithi173 to 205771776: error parsing value: Value '205771776' is below minimum 939524096 2023-12-12T10:09:08.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:07 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi042 to 197.2M 2023-12-12T10:09:08.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:07 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi111 to 196.2M 2023-12-12T10:09:08.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:07 smithi173 ceph-mon[103308]: Unable to set osd_memory_target on smithi111 to 205768192: error parsing value: Value '205768192' is below minimum 939524096 2023-12-12T10:09:08.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:07 smithi173 ceph-mon[103308]: Unable to set osd_memory_target on smithi042 to 206877081: error parsing value: Value '206877081' is below minimum 939524096 2023-12-12T10:09:08.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:07 smithi173 ceph-mon[103308]: Adjusting osd_memory_target on smithi173 to 196.2M 2023-12-12T10:09:08.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:07 smithi173 ceph-mon[103308]: Unable to set osd_memory_target on smithi173 to 205771776: error parsing value: Value '205771776' is below minimum 939524096 2023-12-12T10:09:08.551 INFO:tasks.check_counter:Daemon mds.h mds.exported=1960 2023-12-12T10:09:08.552 INFO:tasks.check_counter:Daemon mds.h mds.imported=1784 2023-12-12T10:09:08.552 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=1616 2023-12-12T10:09:08.552 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=3873 2023-12-12T10:09:08.552 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-12-12T10:09:08.552 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-12-12T10:09:08.552 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-12-12T10:09:08.552 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=68 2023-12-12T10:09:08.552 DEBUG:tasks.check_counter:Getting stats from k 2023-12-12T10:09:08.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.k perf dump 2023-12-12T10:09:08.847 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:08 smithi111 ceph-mon[102565]: pgmap v324: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:09:08.847 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:08 smithi111 ceph-mon[102565]: osdmap e99: 12 total, 12 up, 12 in 2023-12-12T10:09:09.158 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:08 smithi042 ceph-mon[98894]: pgmap v324: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:09:09.158 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:08 smithi042 ceph-mon[98894]: osdmap e99: 12 total, 12 up, 12 in 2023-12-12T10:09:09.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:08 smithi173 ceph-mon[103308]: pgmap v324: 129 pgs: 129 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-12-12T10:09:09.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:08 smithi173 ceph-mon[103308]: osdmap e99: 12 total, 12 up, 12 in 2023-12-12T10:09:09.420 INFO:teuthology.orchestra.run.smithi111.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.b/config 2023-12-12T10:09:10.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:10 smithi111 ceph-mon[102565]: pgmap v326: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:09:11.064 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-12-12T10:09:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:09:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 950, 2023-12-12T10:09:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 952, 2023-12-12T10:09:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 38049040, 2023-12-12T10:09:11.064 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 295514, 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 8, 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.092860309, 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.025488363, 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.037439925, 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.022981258, 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 950, 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.049632056, 2023-12-12T10:09:11.065 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000052244 2023-12-12T10:09:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-12T10:09:11.066 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 0 2023-12-12T10:09:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:09:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 459, 2023-12-12T10:09:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 615, 2023-12-12T10:09:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 63033825, 2023-12-12T10:09:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 567800, 2023-12-12T10:09:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 8, 2023-12-12T10:09:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 6, 2023-12-12T10:09:11.067 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.085890520, 2023-12-12T10:09:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.014517054, 2023-12-12T10:09:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.052516895, 2023-12-12T10:09:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.013753700, 2023-12-12T10:09:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 613, 2023-12-12T10:09:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.052012278, 2023-12-12T10:09:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000084848 2023-12-12T10:09:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.068 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-12-12T10:09:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 426912 2023-12-12T10:09:11.069 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:09:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_messages": 1280, 2023-12-12T10:09:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages": 1246, 2023-12-12T10:09:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_bytes": 69030048, 2023-12-12T10:09:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_bytes": 445869, 2023-12-12T10:09:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_created_connections": 5, 2023-12-12T10:09:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_active_connections": 4, 2023-12-12T10:09:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_total_time": 0.181237908, 2023-12-12T10:09:11.070 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_send_time": 0.040178257, 2023-12-12T10:09:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_recv_time": 0.096646876, 2023-12-12T10:09:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_running_fast_dispatch_time": 0.043668706, 2023-12-12T10:09:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 1245, 2023-12-12T10:09:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.100555970, 2023-12-12T10:09:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000080767 2023-12-12T10:09:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:11.071 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_recv_encrypted_bytes": 1602896, 2023-12-12T10:09:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "msgr_send_encrypted_bytes": 128960 2023-12-12T10:09:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "cct": { 2023-12-12T10:09:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "total_workers": 1, 2023-12-12T10:09:11.072 INFO:teuthology.orchestra.run.smithi111.stdout: "unhealthy_workers": 0 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-MDSRank": { 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 3898, 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.116144863, 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000029796 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.073 INFO:teuthology.orchestra.run.smithi111.stdout: "finisher-PurgeQueue": { 2023-12-12T10:09:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "queue_len": 0, 2023-12-12T10:09:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "complete_latency": { 2023-12-12T10:09:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.074 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": { 2023-12-12T10:09:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "request": 0, 2023-12-12T10:09:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "reply": 0, 2023-12-12T10:09:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "reply_latency": { 2023-12-12T10:09:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_reply": 0, 2023-12-12T10:09:11.075 INFO:teuthology.orchestra.run.smithi111.stdout: "forward": 0, 2023-12-12T10:09:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_complete": 0, 2023-12-12T10:09:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_fetch_keys": 0, 2023-12-12T10:09:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_commit": 0, 2023-12-12T10:09:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_split": 0, 2023-12-12T10:09:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_merge": 0, 2023-12-12T10:09:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes": 56868, 2023-12-12T10:09:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_top": 0, 2023-12-12T10:09:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_bottom": 56868, 2023-12-12T10:09:11.076 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pin_tail": 0, 2023-12-12T10:09:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_pinned": 56868, 2023-12-12T10:09:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_expired": 0, 2023-12-12T10:09:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "inodes_with_caps": 0, 2023-12-12T10:09:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": 0, 2023-12-12T10:09:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "subtrees": 1, 2023-12-12T10:09:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse": 0, 2023-12-12T10:09:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_hit": 0, 2023-12-12T10:09:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_forward": 0, 2023-12-12T10:09:11.077 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_discover": 0, 2023-12-12T10:09:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_dir_fetch": 0, 2023-12-12T10:09:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_remote_ino": 0, 2023-12-12T10:09:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "traverse_lock": 0, 2023-12-12T10:09:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "load_cent": 0, 2023-12-12T10:09:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "q": 0, 2023-12-12T10:09:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "exported": 0, 2023-12-12T10:09:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "exported_inodes": 0, 2023-12-12T10:09:11.078 INFO:teuthology.orchestra.run.smithi111.stdout: "imported": 0, 2023-12-12T10:09:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "imported_inodes": 0, 2023-12-12T10:09:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_dir_fetch": 0, 2023-12-12T10:09:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:09:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "openino_peer_discover": 0, 2023-12-12T10:09:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rfiles": 0, 2023-12-12T10:09:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rbytes": 0, 2023-12-12T10:09:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "root_rsnaps": 0, 2023-12-12T10:09:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:09:11.079 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_set_tag": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "scrub_file_inodes": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_inode_file_caps": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_revoke": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_grant": 0, 2023-12-12T10:09:11.080 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_trunc": 0, 2023-12-12T10:09:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-12T10:09:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:09:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps": 0, 2023-12-12T10:09:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_caps_dirty": 0, 2023-12-12T10:09:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_cap_release": 0, 2023-12-12T10:09:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "process_request_cap_release": 0 2023-12-12T10:09:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_cache": { 2023-12-12T10:09:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update": 0, 2023-12-12T10:09:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_update_receipt": 0, 2023-12-12T10:09:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_try_discover": 0, 2023-12-12T10:09:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_send_discover": 0, 2023-12-12T10:09:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "dir_handle_discover": 0, 2023-12-12T10:09:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays": 0, 2023-12-12T10:09:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_delayed": 0, 2023-12-12T10:09:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:09:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_created": 0, 2023-12-12T10:09:11.082 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_enqueued": 0, 2023-12-12T10:09:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_reintegrated": 0, 2023-12-12T10:09:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "strays_migrated": 0, 2023-12-12T10:09:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_processing": 0, 2023-12-12T10:09:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:09:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:09:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_started": 0, 2023-12-12T10:09:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "recovery_completed": 0, 2023-12-12T10:09:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:09:11.083 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_exportdir": 0, 2023-12-12T10:09:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_flush": 0, 2023-12-12T10:09:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragmentdir": 0, 2023-12-12T10:09:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_fragstats": 0, 2023-12-12T10:09:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "ireq_inodestats": 0 2023-12-12T10:09:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_log": { 2023-12-12T10:09:11.084 INFO:teuthology.orchestra.run.smithi111.stdout: "evlrg": 0, 2023-12-12T10:09:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "evadd": 0, 2023-12-12T10:09:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "evex": 0, 2023-12-12T10:09:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "evtrm": 0, 2023-12-12T10:09:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "ev": 56372, 2023-12-12T10:09:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "evexg": 0, 2023-12-12T10:09:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "evexd": 0, 2023-12-12T10:09:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "segadd": 0, 2023-12-12T10:09:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "segex": 0, 2023-12-12T10:09:11.085 INFO:teuthology.orchestra.run.smithi111.stdout: "segtrm": 0, 2023-12-12T10:09:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "seg": 78, 2023-12-12T10:09:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "segmjr": 6, 2023-12-12T10:09:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "segexg": 0, 2023-12-12T10:09:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "segexd": 0, 2023-12-12T10:09:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "expos": 4194304, 2023-12-12T10:09:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "wrpos": 172017901, 2023-12-12T10:09:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "rdpos": 172017853, 2023-12-12T10:09:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "jlat": { 2023-12-12T10:09:11.086 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "replayed": 56372 2023-12-12T10:09:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_mem": { 2023-12-12T10:09:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "ino": 25067, 2023-12-12T10:09:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "ino+": 80351, 2023-12-12T10:09:11.087 INFO:teuthology.orchestra.run.smithi111.stdout: "ino-": 55284, 2023-12-12T10:09:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dir": 1035, 2023-12-12T10:09:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dir+": 2514, 2023-12-12T10:09:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dir-": 1479, 2023-12-12T10:09:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dn": 56868, 2023-12-12T10:09:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dn+": 134860, 2023-12-12T10:09:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "dn-": 77992, 2023-12-12T10:09:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "cap": 0, 2023-12-12T10:09:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "cap+": 0, 2023-12-12T10:09:11.088 INFO:teuthology.orchestra.run.smithi111.stdout: "cap-": 0, 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "rss": 180872, 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "heap": 207228 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_server": { 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_client_request": 0, 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "dispatch_server_request": 0, 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_request": 0, 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_client_session": 0, 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "handle_peer_request": 0, 2023-12-12T10:09:11.089 INFO:teuthology.orchestra.run.smithi111.stdout: "req_create_latency": { 2023-12-12T10:09:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getattr_latency": { 2023-12-12T10:09:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.090 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getfilelock_latency": { 2023-12-12T10:09:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.091 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "req_link_latency": { 2023-12-12T10:09:11.093 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookup_latency": { 2023-12-12T10:09:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.094 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookuphash_latency": { 2023-12-12T10:09:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupino_latency": { 2023-12-12T10:09:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.095 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupname_latency": { 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupparent_latency": { 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.096 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lookupsnap_latency": { 2023-12-12T10:09:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "req_lssnap_latency": { 2023-12-12T10:09:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.097 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mkdir_latency": { 2023-12-12T10:09:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.098 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mknod_latency": { 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "req_mksnap_latency": { 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.099 INFO:teuthology.orchestra.run.smithi111.stdout: "req_open_latency": { 2023-12-12T10:09:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "req_readdir_latency": { 2023-12-12T10:09:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.100 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rename_latency": { 2023-12-12T10:09:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "req_renamesnap_latency": { 2023-12-12T10:09:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.101 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "req_snapdiff_latency": { 2023-12-12T10:09:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmdir_latency": { 2023-12-12T10:09:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.102 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmsnap_latency": { 2023-12-12T10:09:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "req_rmxattr_latency": { 2023-12-12T10:09:11.103 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setattr_latency": { 2023-12-12T10:09:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.104 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setdirlayout_latency": { 2023-12-12T10:09:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setfilelock_latency": { 2023-12-12T10:09:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.105 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setlayout_latency": { 2023-12-12T10:09:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "req_setxattr_latency": { 2023-12-12T10:09:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.106 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "req_symlink_latency": { 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "req_unlink_latency": { 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.107 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:09:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:09:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "req_getvxattr_latency": { 2023-12-12T10:09:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.108 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_sessions": { 2023-12-12T10:09:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "session_count": 1, 2023-12-12T10:09:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "session_add": 3, 2023-12-12T10:09:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "session_remove": 2, 2023-12-12T10:09:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_open": 2, 2023-12-12T10:09:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "sessions_stale": 0, 2023-12-12T10:09:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "total_load": 0, 2023-12-12T10:09:11.109 INFO:teuthology.orchestra.run.smithi111.stdout: "average_load": 0, 2023-12-12T10:09:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "avg_session_uptime": 6258, 2023-12-12T10:09:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "mdthresh_evicted": 0 2023-12-12T10:09:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "mempool": { 2023-12-12T10:09:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:09:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bloom_filter_items": 0, 2023-12-12T10:09:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:09:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:09:11.110 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:09:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:09:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:09:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:09:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:09:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:09:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:09:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:09:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:09:11.111 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_extent_items": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_blob_items": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:09:11.112 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:09:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:09:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_txc_items": 0, 2023-12-12T10:09:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:09:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:09:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:09:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "bluestore_writing_items": 0, 2023-12-12T10:09:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_bytes": 0, 2023-12-12T10:09:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_items": 0, 2023-12-12T10:09:11.113 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:09:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:09:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:09:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:09:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_bytes": 1083499, 2023-12-12T10:09:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_anon_items": 25137, 2023-12-12T10:09:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_bytes": 0, 2023-12-12T10:09:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "buffer_meta_items": 0, 2023-12-12T10:09:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_bytes": 0, 2023-12-12T10:09:11.114 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_items": 0, 2023-12-12T10:09:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:09:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_mapbl_items": 0, 2023-12-12T10:09:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:09:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_pglog_items": 0, 2023-12-12T10:09:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_bytes": 9848, 2023-12-12T10:09:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_items": 190, 2023-12-12T10:09:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:09:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:09:11.115 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_bytes": 0, 2023-12-12T10:09:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "pgmap_items": 0, 2023-12-12T10:09:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_bytes": 97745779, 2023-12-12T10:09:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_co_items": 789750, 2023-12-12T10:09:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_bytes": 0, 2023-12-12T10:09:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_1_items": 0, 2023-12-12T10:09:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_bytes": 0, 2023-12-12T10:09:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "unittest_2_items": 0 2023-12-12T10:09:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.116 INFO:teuthology.orchestra.run.smithi111.stdout: "objecter": { 2023-12-12T10:09:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "op_active": 0, 2023-12-12T10:09:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "op_laggy": 0, 2023-12-12T10:09:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send": 2422, 2023-12-12T10:09:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "op_send_bytes": 21, 2023-12-12T10:09:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "op_resend": 3, 2023-12-12T10:09:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "op_reply": 2422, 2023-12-12T10:09:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "op_latency": { 2023-12-12T10:09:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2422, 2023-12-12T10:09:11.117 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 4.311923119, 2023-12-12T10:09:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.001780315 2023-12-12T10:09:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "op_inflight": 0, 2023-12-12T10:09:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "oplen_avg": { 2023-12-12T10:09:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 2422, 2023-12-12T10:09:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 2423 2023-12-12T10:09:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.118 INFO:teuthology.orchestra.run.smithi111.stdout: "op": 2422, 2023-12-12T10:09:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "op_r": 2422, 2023-12-12T10:09:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "op_w": 0, 2023-12-12T10:09:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "op_rmw": 0, 2023-12-12T10:09:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "op_pg": 0, 2023-12-12T10:09:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_stat": 1540, 2023-12-12T10:09:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_create": 0, 2023-12-12T10:09:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_read": 880, 2023-12-12T10:09:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_write": 0, 2023-12-12T10:09:11.119 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writefull": 0, 2023-12-12T10:09:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_writesame": 0, 2023-12-12T10:09:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_append": 0, 2023-12-12T10:09:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_zero": 0, 2023-12-12T10:09:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_truncate": 0, 2023-12-12T10:09:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_delete": 0, 2023-12-12T10:09:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_mapext": 0, 2023-12-12T10:09:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_sparse_read": 0, 2023-12-12T10:09:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_clonerange": 0, 2023-12-12T10:09:11.120 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_getxattr": 1, 2023-12-12T10:09:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_setxattr": 0, 2023-12-12T10:09:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:09:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_rmxattr": 0, 2023-12-12T10:09:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:09:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_call": 0, 2023-12-12T10:09:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_watch": 0, 2023-12-12T10:09:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_notify": 0, 2023-12-12T10:09:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:09:11.121 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls": 0, 2023-12-12T10:09:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:09:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "osdop_other": 0, 2023-12-12T10:09:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_active": 0, 2023-12-12T10:09:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_send": 0, 2023-12-12T10:09:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_resend": 0, 2023-12-12T10:09:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "linger_ping": 0, 2023-12-12T10:09:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_active": 0, 2023-12-12T10:09:11.122 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_send": 0, 2023-12-12T10:09:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "poolop_resend": 0, 2023-12-12T10:09:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_active": 0, 2023-12-12T10:09:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_send": 0, 2023-12-12T10:09:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "poolstat_resend": 0, 2023-12-12T10:09:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_active": 0, 2023-12-12T10:09:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_send": 0, 2023-12-12T10:09:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "statfs_resend": 0, 2023-12-12T10:09:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "command_active": 0, 2023-12-12T10:09:11.123 INFO:teuthology.orchestra.run.smithi111.stdout: "command_send": 0, 2023-12-12T10:09:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "command_resend": 0, 2023-12-12T10:09:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "map_epoch": 99, 2023-12-12T10:09:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "map_full": 0, 2023-12-12T10:09:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "map_inc": 12, 2023-12-12T10:09:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_sessions": 10, 2023-12-12T10:09:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_open": 10, 2023-12-12T10:09:11.124 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_session_close": 0, 2023-12-12T10:09:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_laggy": 0, 2023-12-12T10:09:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_wr": 0, 2023-12-12T10:09:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_rd": 2, 2023-12-12T10:09:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_del": 0 2023-12-12T10:09:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "oft": { 2023-12-12T10:09:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_objs": 0, 2023-12-12T10:09:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:09:11.125 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_updates": 0, 2023-12-12T10:09:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "omap_total_removes": 0 2023-12-12T10:09:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "osd-slow-ops": { 2023-12-12T10:09:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "slow_ops_count": 0 2023-12-12T10:09:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "purge_queue": { 2023-12-12T10:09:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops": 0, 2023-12-12T10:09:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_ops_high_water": 0, 2023-12-12T10:09:11.126 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing": 0, 2023-12-12T10:09:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executing_high_water": 0, 2023-12-12T10:09:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_executed": 0, 2023-12-12T10:09:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "pq_item_in_journal": 0 2023-12-12T10:09:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:09:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2023-12-12T10:09:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:11.127 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 2685, 2023-12-12T10:09:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 169855030, 2023-12-12T10:09:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 2685, 2023-12-12T10:09:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2685, 2023-12-12T10:09:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 169855030, 2023-12-12T10:09:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:11.128 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_bytes": { 2023-12-12T10:09:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 104857600, 2023-12-12T10:09:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:11.129 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2422, 2023-12-12T10:09:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 167825119, 2023-12-12T10:09:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2422, 2023-12-12T10:09:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 167825119, 2023-12-12T10:09:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:11.130 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-objecter_ops": { 2023-12-12T10:09:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 1024, 2023-12-12T10:09:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:11.131 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 2422, 2023-12-12T10:09:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 2422, 2023-12-12T10:09:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 2422, 2023-12-12T10:09:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 2422, 2023-12-12T10:09:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:11.132 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:11.133 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2023-12-12T10:09:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2023-12-12T10:09:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:11.134 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-12-12T10:09:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "throttle-write_buf_throttle-0x5621ba390960": { 2023-12-12T10:09:11.135 INFO:teuthology.orchestra.run.smithi111.stdout: "val": 0, 2023-12-12T10:09:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "max": 3758096384, 2023-12-12T10:09:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "get_started": 0, 2023-12-12T10:09:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "get": 0, 2023-12-12T10:09:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "get_sum": 0, 2023-12-12T10:09:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "take": 0, 2023-12-12T10:09:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "take_sum": 0, 2023-12-12T10:09:11.136 INFO:teuthology.orchestra.run.smithi111.stdout: "put": 0, 2023-12-12T10:09:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "put_sum": 0, 2023-12-12T10:09:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "wait": { 2023-12-12T10:09:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "avgcount": 0, 2023-12-12T10:09:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "sum": 0.000000000, 2023-12-12T10:09:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: "avgtime": 0.000000000 2023-12-12T10:09:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:11.137 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-12-12T10:09:11.137 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-12-12T10:09:11.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:10 smithi042 ceph-mon[98894]: pgmap v326: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:09:11.325 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:10 smithi173 ceph-mon[103308]: pgmap v326: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:09:11.452 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-12-12T10:09:11.452 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-12-12T10:09:11.452 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2023-12-12T10:09:11.452 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2023-12-12T10:09:11.453 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2023-12-12T10:09:11.453 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-12-12T10:09:11.453 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-12-12T10:09:11.453 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2023-12-12T10:09:11.453 DEBUG:tasks.check_counter:Getting stats from c 2023-12-12T10:09:11.453 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.c perf dump 2023-12-12T10:09:12.365 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T10:09:12.885 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:12 smithi173 ceph-mon[103308]: pgmap v327: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:09:13.155 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:12 smithi042 ceph-mon[98894]: pgmap v327: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:09:13.253 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:12 smithi111 ceph-mon[102565]: pgmap v327: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:09:15.156 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:14 smithi042 ceph-mon[98894]: pgmap v328: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:09:15.252 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:14 smithi111 ceph-mon[102565]: pgmap v328: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:09:15.323 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:14 smithi173 ceph-mon[103308]: pgmap v328: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 717 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 28 op/s 2023-12-12T10:09:15.784 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-12-12T10:09:15.784 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:09:15.784 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 17295, 2023-12-12T10:09:15.784 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 18323, 2023-12-12T10:09:15.784 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 6352221, 2023-12-12T10:09:15.784 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 75567980, 2023-12-12T10:09:15.785 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 14, 2023-12-12T10:09:15.785 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 6, 2023-12-12T10:09:15.785 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 1.394854560, 2023-12-12T10:09:15.785 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.493221588, 2023-12-12T10:09:15.785 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.542455434, 2023-12-12T10:09:15.785 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.426806688, 2023-12-12T10:09:15.785 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:15.785 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 18321, 2023-12-12T10:09:15.785 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 4.418105118, 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000241149 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 14530, 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.003009169, 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000207 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1626672, 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 134512 2023-12-12T10:09:15.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.787 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:09:15.787 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 84027, 2023-12-12T10:09:15.787 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 122710, 2023-12-12T10:09:15.787 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 104846215, 2023-12-12T10:09:15.787 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 199434386, 2023-12-12T10:09:15.787 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 15, 2023-12-12T10:09:15.787 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 6, 2023-12-12T10:09:15.787 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 5.130755891, 2023-12-12T10:09:15.787 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 2.161563612, 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 24.122991076, 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.499539948, 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 122708, 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 29.488462643, 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000240314 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 161269, 2023-12-12T10:09:15.788 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.031019210, 2023-12-12T10:09:15.789 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000192 2023-12-12T10:09:15.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.789 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-12T10:09:15.789 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 429456 2023-12-12T10:09:15.789 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.789 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:09:15.789 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 30380, 2023-12-12T10:09:15.789 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 36243, 2023-12-12T10:09:15.789 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 21994364, 2023-12-12T10:09:15.790 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 188401109, 2023-12-12T10:09:15.790 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 9, 2023-12-12T10:09:15.790 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 5, 2023-12-12T10:09:15.790 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 2.036414149, 2023-12-12T10:09:15.790 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.736366829, 2023-12-12T10:09:15.790 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 1.034376986, 2023-12-12T10:09:15.791 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.458898544, 2023-12-12T10:09:15.791 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:15.791 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 36242, 2023-12-12T10:09:15.791 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 9.985511196, 2023-12-12T10:09:15.792 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000275523 2023-12-12T10:09:15.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.792 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:15.792 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 40984, 2023-12-12T10:09:15.792 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.011537538, 2023-12-12T10:09:15.792 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000281 2023-12-12T10:09:15.792 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.792 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-12T10:09:15.793 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 448 2023-12-12T10:09:15.793 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.793 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2023-12-12T10:09:15.793 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2023-12-12T10:09:15.793 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2023-12-12T10:09:15.794 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.794 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2023-12-12T10:09:15.794 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2023-12-12T10:09:15.794 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2023-12-12T10:09:15.794 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 5442, 2023-12-12T10:09:15.794 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 34.537533207, 2023-12-12T10:09:15.794 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.006346477 2023-12-12T10:09:15.794 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:15.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.795 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2023-12-12T10:09:15.795 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2023-12-12T10:09:15.795 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2023-12-12T10:09:15.795 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 20724, 2023-12-12T10:09:15.795 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 7.403455553, 2023-12-12T10:09:15.795 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000357240 2023-12-12T10:09:15.795 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:15.795 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.796 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2023-12-12T10:09:15.796 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 35722, 2023-12-12T10:09:15.796 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 28439, 2023-12-12T10:09:15.796 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2023-12-12T10:09:15.796 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 28439, 2023-12-12T10:09:15.796 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 91.829530064, 2023-12-12T10:09:15.796 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.003228999 2023-12-12T10:09:15.796 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.796 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2023-12-12T10:09:15.797 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 7282, 2023-12-12T10:09:15.797 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 338, 2023-12-12T10:09:15.797 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2023-12-12T10:09:15.797 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 283, 2023-12-12T10:09:15.797 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 34, 2023-12-12T10:09:15.797 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2023-12-12T10:09:15.797 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 45275, 2023-12-12T10:09:15.797 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 140, 2023-12-12T10:09:15.797 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 107, 2023-12-12T10:09:15.798 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 45028, 2023-12-12T10:09:15.798 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 45073, 2023-12-12T10:09:15.798 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 74660, 2023-12-12T10:09:15.798 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2023-12-12T10:09:15.799 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2023-12-12T10:09:15.799 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 4, 2023-12-12T10:09:15.799 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 50264, 2023-12-12T10:09:15.799 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 42281, 2023-12-12T10:09:15.799 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 7100, 2023-12-12T10:09:15.799 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 7, 2023-12-12T10:09:15.799 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 161, 2023-12-12T10:09:15.799 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2023-12-12T10:09:15.800 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2023-12-12T10:09:15.800 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 1000, 2023-12-12T10:09:15.800 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 1, 2023-12-12T10:09:15.800 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 1715, 2023-12-12T10:09:15.800 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 77607, 2023-12-12T10:09:15.800 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 1876, 2023-12-12T10:09:15.801 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 106743, 2023-12-12T10:09:15.801 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2023-12-12T10:09:15.801 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:09:15.801 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2023-12-12T10:09:15.801 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 4, 2023-12-12T10:09:15.801 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 246, 2023-12-12T10:09:15.801 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2023-12-12T10:09:15.802 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:09:15.802 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2023-12-12T10:09:15.802 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:09:15.802 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:09:15.802 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:09:15.802 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:09:15.802 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:09:15.802 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2023-12-12T10:09:15.803 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 2, 2023-12-12T10:09:15.803 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 219, 2023-12-12T10:09:15.803 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 15570, 2023-12-12T10:09:15.803 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2023-12-12T10:09:15.803 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-12T10:09:15.803 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:09:15.804 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 12225, 2023-12-12T10:09:15.804 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 12004, 2023-12-12T10:09:15.804 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 248, 2023-12-12T10:09:15.804 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 25215 2023-12-12T10:09:15.804 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 1835, 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 4052, 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 2028, 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 3075, 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 2217, 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 2, 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:09:15.805 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 11802, 2023-12-12T10:09:15.806 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 11800, 2023-12-12T10:09:15.806 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2023-12-12T10:09:15.806 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2023-12-12T10:09:15.806 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2023-12-12T10:09:15.806 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:09:15.806 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:09:15.806 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2023-12-12T10:09:15.807 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2023-12-12T10:09:15.807 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:09:15.807 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 1970, 2023-12-12T10:09:15.807 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2023-12-12T10:09:15.807 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 34, 2023-12-12T10:09:15.807 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2023-12-12T10:09:15.807 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2023-12-12T10:09:15.808 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.808 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2023-12-12T10:09:15.808 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 3, 2023-12-12T10:09:15.808 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 56372, 2023-12-12T10:09:15.808 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2023-12-12T10:09:15.808 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2023-12-12T10:09:15.809 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 56372, 2023-12-12T10:09:15.809 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2023-12-12T10:09:15.809 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2023-12-12T10:09:15.809 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 78, 2023-12-12T10:09:15.809 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2023-12-12T10:09:15.810 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2023-12-12T10:09:15.810 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 78, 2023-12-12T10:09:15.810 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 6, 2023-12-12T10:09:15.810 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2023-12-12T10:09:15.810 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2023-12-12T10:09:15.810 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2023-12-12T10:09:15.810 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 172017901, 2023-12-12T10:09:15.811 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2023-12-12T10:09:15.811 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2023-12-12T10:09:15.811 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4565, 2023-12-12T10:09:15.811 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 404.151442751, 2023-12-12T10:09:15.811 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.088532627 2023-12-12T10:09:15.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.811 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2023-12-12T10:09:15.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.811 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2023-12-12T10:09:15.812 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 25276, 2023-12-12T10:09:15.812 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 81292, 2023-12-12T10:09:15.812 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 56016, 2023-12-12T10:09:15.812 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 1046, 2023-12-12T10:09:15.812 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 2800, 2023-12-12T10:09:15.812 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 1754, 2023-12-12T10:09:15.812 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 45275, 2023-12-12T10:09:15.812 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 131735, 2023-12-12T10:09:15.812 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 86460, 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 19633, 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 19633, 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 257620, 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 199036 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 38847, 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 286, 2023-12-12T10:09:15.813 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 35722, 2023-12-12T10:09:15.814 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 17, 2023-12-12T10:09:15.814 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 1289, 2023-12-12T10:09:15.814 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2023-12-12T10:09:15.817 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3307, 2023-12-12T10:09:15.817 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 5.368432595, 2023-12-12T10:09:15.817 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001623354 2023-12-12T10:09:15.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.818 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2023-12-12T10:09:15.818 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3, 2023-12-12T10:09:15.818 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.007885435, 2023-12-12T10:09:15.818 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002628478 2023-12-12T10:09:15.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.819 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2023-12-12T10:09:15.819 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.819 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.819 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.820 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.820 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2023-12-12T10:09:15.820 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.820 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.820 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.821 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.821 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2023-12-12T10:09:15.821 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 11632, 2023-12-12T10:09:15.821 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 22.523133339, 2023-12-12T10:09:15.822 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001936307 2023-12-12T10:09:15.822 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.822 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2023-12-12T10:09:15.822 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.822 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.822 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.823 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.823 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2023-12-12T10:09:15.823 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.823 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.824 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.824 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2023-12-12T10:09:15.824 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.824 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.824 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.825 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.825 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2023-12-12T10:09:15.825 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.825 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.825 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.826 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2023-12-12T10:09:15.826 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.826 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.826 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.826 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.827 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2023-12-12T10:09:15.827 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.827 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.827 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.827 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2023-12-12T10:09:15.827 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.828 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.828 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.828 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.828 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2023-12-12T10:09:15.829 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.829 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.829 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.829 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2023-12-12T10:09:15.829 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.829 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.829 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.829 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.830 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2023-12-12T10:09:15.830 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.830 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.830 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.831 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2023-12-12T10:09:15.831 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1695, 2023-12-12T10:09:15.831 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 9.256592442, 2023-12-12T10:09:15.831 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.005461116 2023-12-12T10:09:15.831 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.831 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2023-12-12T10:09:15.832 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.832 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.832 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.832 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2023-12-12T10:09:15.832 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.832 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.832 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.833 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2023-12-12T10:09:15.833 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.833 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.833 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.834 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2023-12-12T10:09:15.834 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2023-12-12T10:09:15.834 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.100747585, 2023-12-12T10:09:15.834 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.050373792 2023-12-12T10:09:15.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.834 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2023-12-12T10:09:15.834 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.834 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.835 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2023-12-12T10:09:15.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.835 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.835 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2023-12-12T10:09:15.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2023-12-12T10:09:15.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.837 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.837 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2023-12-12T10:09:15.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.837 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.837 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2023-12-12T10:09:15.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.838 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.838 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2023-12-12T10:09:15.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.838 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.839 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2023-12-12T10:09:15.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 11800, 2023-12-12T10:09:15.839 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 54.572738668, 2023-12-12T10:09:15.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.004624808 2023-12-12T10:09:15.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.840 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:09:15.840 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:09:15.840 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2023-12-12T10:09:15.840 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.840 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.840 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.840 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:15.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.840 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2023-12-12T10:09:15.841 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 1, 2023-12-12T10:09:15.841 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 3, 2023-12-12T10:09:15.841 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 2, 2023-12-12T10:09:15.841 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 1, 2023-12-12T10:09:15.841 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2023-12-12T10:09:15.841 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 10157, 2023-12-12T10:09:15.842 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 10157, 2023-12-12T10:09:15.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 644, 2023-12-12T10:09:15.842 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2023-12-12T10:09:15.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.842 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2023-12-12T10:09:15.842 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:09:15.842 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2023-12-12T10:09:15.842 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:09:15.843 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:09:15.843 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:09:15.843 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:09:15.843 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:09:15.843 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:09:15.843 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:09:15.843 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:09:15.843 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:09:15.844 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:09:15.844 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:09:15.844 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:09:15.844 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:09:15.844 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2023-12-12T10:09:15.845 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:09:15.845 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2023-12-12T10:09:15.845 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:09:15.845 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:09:15.846 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:09:15.846 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:09:15.846 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:09:15.846 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:09:15.847 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:09:15.847 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2023-12-12T10:09:15.847 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:09:15.847 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:09:15.848 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:09:15.848 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2023-12-12T10:09:15.848 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2023-12-12T10:09:15.848 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2023-12-12T10:09:15.848 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:09:15.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:09:15.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:09:15.849 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:09:15.849 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 1187157, 2023-12-12T10:09:15.850 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 25379, 2023-12-12T10:09:15.850 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2023-12-12T10:09:15.850 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2023-12-12T10:09:15.850 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2023-12-12T10:09:15.850 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2023-12-12T10:09:15.851 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:09:15.851 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2023-12-12T10:09:15.851 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:09:15.851 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2023-12-12T10:09:15.851 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 9848, 2023-12-12T10:09:15.851 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 190, 2023-12-12T10:09:15.851 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:09:15.851 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:09:15.851 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2023-12-12T10:09:15.852 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2023-12-12T10:09:15.852 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 90693905, 2023-12-12T10:09:15.852 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 825233, 2023-12-12T10:09:15.852 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2023-12-12T10:09:15.852 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2023-12-12T10:09:15.853 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2023-12-12T10:09:15.853 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2023-12-12T10:09:15.853 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.853 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2023-12-12T10:09:15.853 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2023-12-12T10:09:15.853 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2023-12-12T10:09:15.853 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 30010, 2023-12-12T10:09:15.853 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 172663123, 2023-12-12T10:09:15.853 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 30, 2023-12-12T10:09:15.854 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 30010, 2023-12-12T10:09:15.854 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2023-12-12T10:09:15.855 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 30010, 2023-12-12T10:09:15.855 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 334.601609803, 2023-12-12T10:09:15.855 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.011149670 2023-12-12T10:09:15.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.855 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2023-12-12T10:09:15.855 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2023-12-12T10:09:15.855 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 30010, 2023-12-12T10:09:15.855 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 30705 2023-12-12T10:09:15.855 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.856 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 30010, 2023-12-12T10:09:15.856 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 1058, 2023-12-12T10:09:15.856 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 28952, 2023-12-12T10:09:15.856 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2023-12-12T10:09:15.856 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2023-12-12T10:09:15.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2023-12-12T10:09:15.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2023-12-12T10:09:15.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 257, 2023-12-12T10:09:15.856 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 4823, 2023-12-12T10:09:15.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 108, 2023-12-12T10:09:15.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2023-12-12T10:09:15.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2023-12-12T10:09:15.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2023-12-12T10:09:15.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2023-12-12T10:09:15.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 23686, 2023-12-12T10:09:15.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2023-12-12T10:09:15.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2023-12-12T10:09:15.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2023-12-12T10:09:15.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2023-12-12T10:09:15.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2023-12-12T10:09:15.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:09:15.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2023-12-12T10:09:15.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:09:15.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2023-12-12T10:09:15.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2023-12-12T10:09:15.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2023-12-12T10:09:15.859 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:09:15.859 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2023-12-12T10:09:15.859 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:09:15.859 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2023-12-12T10:09:15.859 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2023-12-12T10:09:15.859 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2023-12-12T10:09:15.859 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2023-12-12T10:09:15.859 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2023-12-12T10:09:15.860 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2023-12-12T10:09:15.860 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2023-12-12T10:09:15.860 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2023-12-12T10:09:15.860 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2023-12-12T10:09:15.860 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2023-12-12T10:09:15.860 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2023-12-12T10:09:15.860 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2023-12-12T10:09:15.861 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2023-12-12T10:09:15.861 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2023-12-12T10:09:15.861 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2023-12-12T10:09:15.861 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2023-12-12T10:09:15.861 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2023-12-12T10:09:15.861 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 99, 2023-12-12T10:09:15.861 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2023-12-12T10:09:15.862 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 12, 2023-12-12T10:09:15.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 12, 2023-12-12T10:09:15.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 12, 2023-12-12T10:09:15.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2023-12-12T10:09:15.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2023-12-12T10:09:15.862 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 633, 2023-12-12T10:09:15.863 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 1139, 2023-12-12T10:09:15.863 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 29 2023-12-12T10:09:15.863 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.863 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2023-12-12T10:09:15.863 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2023-12-12T10:09:15.863 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:09:15.864 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2023-12-12T10:09:15.864 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2023-12-12T10:09:15.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.864 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2023-12-12T10:09:15.864 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2023-12-12T10:09:15.864 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.864 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2023-12-12T10:09:15.864 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2023-12-12T10:09:15.864 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 10, 2023-12-12T10:09:15.865 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2023-12-12T10:09:15.865 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 5, 2023-12-12T10:09:15.865 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 11800, 2023-12-12T10:09:15.865 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2023-12-12T10:09:15.865 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.865 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:09:15.865 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:15.865 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2023-12-12T10:09:15.865 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:15.866 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 131698, 2023-12-12T10:09:15.866 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 121322013, 2023-12-12T10:09:15.866 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:15.866 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 131698, 2023-12-12T10:09:15.866 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:15.866 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:15.866 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 131698, 2023-12-12T10:09:15.866 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 121322013, 2023-12-12T10:09:15.866 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:15.867 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.867 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.867 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.867 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:15.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.867 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2023-12-12T10:09:15.868 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:15.868 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2023-12-12T10:09:15.868 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:15.869 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:15.869 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:15.869 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:15.869 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:15.869 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 30010, 2023-12-12T10:09:15.870 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 174081720, 2023-12-12T10:09:15.870 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 30010, 2023-12-12T10:09:15.870 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 174081720, 2023-12-12T10:09:15.871 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:15.872 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.872 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.872 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.872 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:15.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.873 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2023-12-12T10:09:15.873 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:15.873 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2023-12-12T10:09:15.874 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:15.874 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:15.874 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:15.875 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:15.875 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:15.875 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 30010, 2023-12-12T10:09:15.875 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 30010, 2023-12-12T10:09:15.875 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 30010, 2023-12-12T10:09:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 30010, 2023-12-12T10:09:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.876 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.877 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:09:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2023-12-12T10:09:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:15.878 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 11800, 2023-12-12T10:09:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 1437336, 2023-12-12T10:09:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 11800, 2023-12-12T10:09:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:15.879 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 257, 2023-12-12T10:09:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1437336, 2023-12-12T10:09:15.880 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.881 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x5586669aa6e0": { 2023-12-12T10:09:15.882 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2023-12-12T10:09:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 56372, 2023-12-12T10:09:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 167823597, 2023-12-12T10:09:15.883 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 56372, 2023-12-12T10:09:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:15.884 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 4565, 2023-12-12T10:09:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 167823597, 2023-12-12T10:09:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:15.885 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:15.886 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:15.886 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-12-12T10:09:16.326 INFO:tasks.check_counter:Daemon mds.c mds.exported=1715 2023-12-12T10:09:16.326 INFO:tasks.check_counter:Daemon mds.c mds.imported=1876 2023-12-12T10:09:16.326 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=1835 2023-12-12T10:09:16.327 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=4052 2023-12-12T10:09:16.327 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2023-12-12T10:09:16.327 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-12-12T10:09:16.327 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-12-12T10:09:16.327 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=34 2023-12-12T10:09:16.327 DEBUG:tasks.check_counter:Getting stats from f 2023-12-12T10:09:16.327 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.f perf dump 2023-12-12T10:09:17.221 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T10:09:17.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:16 smithi111 ceph-mon[102565]: pgmap v329: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:09:17.324 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:16 smithi173 ceph-mon[103308]: pgmap v329: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:09:17.405 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:16 smithi042 ceph-mon[98894]: pgmap v329: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:09:18.806 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-12-12T10:09:18.806 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:09:18.806 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 38186, 2023-12-12T10:09:18.806 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 42762, 2023-12-12T10:09:18.807 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 65941820, 2023-12-12T10:09:18.807 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 67716156, 2023-12-12T10:09:18.807 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 9, 2023-12-12T10:09:18.808 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2023-12-12T10:09:18.808 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 2.348096579, 2023-12-12T10:09:18.808 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.935058479, 2023-12-12T10:09:18.808 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 1.564289760, 2023-12-12T10:09:18.808 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.299487818, 2023-12-12T10:09:18.808 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:18.808 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 42763, 2023-12-12T10:09:18.808 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 15.612223079, 2023-12-12T10:09:18.809 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000365087 2023-12-12T10:09:18.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.809 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:18.809 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 64318, 2023-12-12T10:09:18.809 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.015558669, 2023-12-12T10:09:18.809 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000241 2023-12-12T10:09:18.809 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.809 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1614896, 2023-12-12T10:09:18.809 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 134624 2023-12-12T10:09:18.810 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.810 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:09:18.810 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 44972, 2023-12-12T10:09:18.810 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 76947, 2023-12-12T10:09:18.810 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 14482138, 2023-12-12T10:09:18.810 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 64331814, 2023-12-12T10:09:18.810 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 19, 2023-12-12T10:09:18.810 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 8, 2023-12-12T10:09:18.810 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 2.950068389, 2023-12-12T10:09:18.811 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 1.116330083, 2023-12-12T10:09:18.811 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 23.619333371, 2023-12-12T10:09:18.811 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.499014570, 2023-12-12T10:09:18.811 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:18.811 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 76942, 2023-12-12T10:09:18.811 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 17.770410536, 2023-12-12T10:09:18.811 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000230958 2023-12-12T10:09:18.811 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.811 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:18.812 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 82296, 2023-12-12T10:09:18.812 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.016279980, 2023-12-12T10:09:18.812 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000197 2023-12-12T10:09:18.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.812 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1248, 2023-12-12T10:09:18.812 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 430848 2023-12-12T10:09:18.812 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.812 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:09:18.812 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 19513, 2023-12-12T10:09:18.813 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 18466, 2023-12-12T10:09:18.813 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 41187848, 2023-12-12T10:09:18.813 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 65445533, 2023-12-12T10:09:18.813 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 10, 2023-12-12T10:09:18.813 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 7, 2023-12-12T10:09:18.813 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 1.487525397, 2023-12-12T10:09:18.813 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.554531535, 2023-12-12T10:09:18.813 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.543026572, 2023-12-12T10:09:18.813 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.459620231, 2023-12-12T10:09:18.814 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:18.814 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 18465, 2023-12-12T10:09:18.814 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 4.843929203, 2023-12-12T10:09:18.814 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000262330 2023-12-12T10:09:18.814 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.814 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:18.814 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 14547, 2023-12-12T10:09:18.814 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.003792172, 2023-12-12T10:09:18.814 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000260 2023-12-12T10:09:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 4720, 2023-12-12T10:09:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 6112 2023-12-12T10:09:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2023-12-12T10:09:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2023-12-12T10:09:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2023-12-12T10:09:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.815 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2023-12-12T10:09:18.816 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2023-12-12T10:09:18.816 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2023-12-12T10:09:18.816 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3428, 2023-12-12T10:09:18.816 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 29.726966569, 2023-12-12T10:09:18.816 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.008671810 2023-12-12T10:09:18.816 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:18.816 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.816 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2023-12-12T10:09:18.816 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 19046, 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 7.446245190, 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000390961 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 24045, 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 23716, 2023-12-12T10:09:18.817 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2023-12-12T10:09:18.818 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 23716, 2023-12-12T10:09:18.818 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 75.057453767, 2023-12-12T10:09:18.818 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.003164844 2023-12-12T10:09:18.818 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.818 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2023-12-12T10:09:18.818 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 329, 2023-12-12T10:09:18.819 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 147, 2023-12-12T10:09:18.819 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2023-12-12T10:09:18.819 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 243, 2023-12-12T10:09:18.819 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 29, 2023-12-12T10:09:18.819 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2023-12-12T10:09:18.819 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 20, 2023-12-12T10:09:18.819 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 2, 2023-12-12T10:09:18.819 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 8, 2023-12-12T10:09:18.819 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 10, 2023-12-12T10:09:18.820 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 15, 2023-12-12T10:09:18.820 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 47410, 2023-12-12T10:09:18.820 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 2, 2023-12-12T10:09:18.820 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 2, 2023-12-12T10:09:18.820 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 5, 2023-12-12T10:09:18.820 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 38536, 2023-12-12T10:09:18.820 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 37832, 2023-12-12T10:09:18.820 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 88, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 115, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 145, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 1328, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 53972, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 1125, 2023-12-12T10:09:18.821 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 40108, 2023-12-12T10:09:18.822 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2023-12-12T10:09:18.822 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:09:18.822 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2023-12-12T10:09:18.822 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 4, 2023-12-12T10:09:18.822 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 246, 2023-12-12T10:09:18.822 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2023-12-12T10:09:18.822 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:09:18.822 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 43, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 12729, 2023-12-12T10:09:18.823 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2023-12-12T10:09:18.824 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-12T10:09:18.824 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:09:18.824 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 9481, 2023-12-12T10:09:18.824 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 9433, 2023-12-12T10:09:18.824 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 259, 2023-12-12T10:09:18.824 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 22234 2023-12-12T10:09:18.824 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.824 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2023-12-12T10:09:18.824 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 738, 2023-12-12T10:09:18.825 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 6209, 2023-12-12T10:09:18.825 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 3109, 2023-12-12T10:09:18.825 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 4868, 2023-12-12T10:09:18.825 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 1080, 2023-12-12T10:09:18.825 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2023-12-12T10:09:18.825 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2023-12-12T10:09:18.825 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:09:18.825 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 10849, 2023-12-12T10:09:18.825 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 10849, 2023-12-12T10:09:18.826 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2023-12-12T10:09:18.826 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2023-12-12T10:09:18.826 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2023-12-12T10:09:18.826 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:09:18.826 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:09:18.826 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2023-12-12T10:09:18.826 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2023-12-12T10:09:18.826 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:09:18.826 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 1629, 2023-12-12T10:09:18.827 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2023-12-12T10:09:18.827 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 29, 2023-12-12T10:09:18.827 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2023-12-12T10:09:18.827 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2023-12-12T10:09:18.827 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.827 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2023-12-12T10:09:18.828 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 2, 2023-12-12T10:09:18.828 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 47777, 2023-12-12T10:09:18.828 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2023-12-12T10:09:18.828 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2023-12-12T10:09:18.828 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 47777, 2023-12-12T10:09:18.828 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2023-12-12T10:09:18.828 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2023-12-12T10:09:18.828 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 67, 2023-12-12T10:09:18.828 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2023-12-12T10:09:18.829 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2023-12-12T10:09:18.829 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 67, 2023-12-12T10:09:18.829 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 5, 2023-12-12T10:09:18.829 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2023-12-12T10:09:18.829 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2023-12-12T10:09:18.829 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2023-12-12T10:09:18.829 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 125706584, 2023-12-12T10:09:18.829 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2023-12-12T10:09:18.829 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2023-12-12T10:09:18.830 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 3183, 2023-12-12T10:09:18.830 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 365.159079430, 2023-12-12T10:09:18.830 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.114721671 2023-12-12T10:09:18.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.830 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2023-12-12T10:09:18.830 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.830 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2023-12-12T10:09:18.831 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 26, 2023-12-12T10:09:18.831 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 41030, 2023-12-12T10:09:18.831 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 41004, 2023-12-12T10:09:18.831 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 24, 2023-12-12T10:09:18.831 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 3623, 2023-12-12T10:09:18.831 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 3599, 2023-12-12T10:09:18.831 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 20, 2023-12-12T10:09:18.831 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 58279, 2023-12-12T10:09:18.831 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 58259, 2023-12-12T10:09:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 2, 2023-12-12T10:09:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 18066, 2023-12-12T10:09:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 18064, 2023-12-12T10:09:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 169788, 2023-12-12T10:09:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 199036 2023-12-12T10:09:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2023-12-12T10:09:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 25100, 2023-12-12T10:09:18.832 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 3, 2023-12-12T10:09:18.833 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 24045, 2023-12-12T10:09:18.833 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 22, 2023-12-12T10:09:18.833 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 676, 2023-12-12T10:09:18.833 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2023-12-12T10:09:18.833 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1727, 2023-12-12T10:09:18.833 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 3.703800732, 2023-12-12T10:09:18.833 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.002144644 2023-12-12T10:09:18.833 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.833 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2023-12-12T10:09:18.834 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 5, 2023-12-12T10:09:18.834 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.004576478, 2023-12-12T10:09:18.834 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000915295 2023-12-12T10:09:18.834 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.834 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2023-12-12T10:09:18.834 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.834 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.835 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2023-12-12T10:09:18.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.835 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.835 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.835 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2023-12-12T10:09:18.835 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 10768, 2023-12-12T10:09:18.836 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 17.929123423, 2023-12-12T10:09:18.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001665037 2023-12-12T10:09:18.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.836 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2023-12-12T10:09:18.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.836 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.836 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.836 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.836 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2023-12-12T10:09:18.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2, 2023-12-12T10:09:18.837 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000359072, 2023-12-12T10:09:18.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000179536 2023-12-12T10:09:18.837 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.837 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2023-12-12T10:09:18.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.837 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.837 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.838 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2023-12-12T10:09:18.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.838 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.838 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.838 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2023-12-12T10:09:18.838 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.839 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.839 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2023-12-12T10:09:18.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.839 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.839 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.839 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.840 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2023-12-12T10:09:18.840 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.840 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.840 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.840 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.840 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2023-12-12T10:09:18.840 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.840 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.840 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.841 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2023-12-12T10:09:18.841 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.841 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.841 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.841 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.841 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2023-12-12T10:09:18.841 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.841 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.842 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2023-12-12T10:09:18.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 365, 2023-12-12T10:09:18.842 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.492904018, 2023-12-12T10:09:18.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001350421 2023-12-12T10:09:18.842 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.842 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2023-12-12T10:09:18.842 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.843 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.843 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.843 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2023-12-12T10:09:18.843 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.843 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.843 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.843 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.844 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2023-12-12T10:09:18.844 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.844 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.844 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.844 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.844 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2023-12-12T10:09:18.844 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.844 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.844 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.845 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2023-12-12T10:09:18.845 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.845 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.845 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.845 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.845 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2023-12-12T10:09:18.845 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.846 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.846 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.846 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2023-12-12T10:09:18.846 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.846 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.846 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.846 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.846 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2023-12-12T10:09:18.847 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.847 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.847 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.847 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2023-12-12T10:09:18.847 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.847 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.847 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.847 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.848 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2023-12-12T10:09:18.848 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.848 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.848 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.848 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.848 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2023-12-12T10:09:18.848 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.848 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.848 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.849 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2023-12-12T10:09:18.849 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.849 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.849 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.849 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.849 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2023-12-12T10:09:18.849 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 10849, 2023-12-12T10:09:18.849 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 52.926690044, 2023-12-12T10:09:18.850 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.004878485 2023-12-12T10:09:18.850 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.850 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:09:18.850 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:09:18.850 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2023-12-12T10:09:18.850 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.850 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.850 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.850 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:18.851 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.851 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2023-12-12T10:09:18.851 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 2, 2023-12-12T10:09:18.851 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 3, 2023-12-12T10:09:18.851 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 1, 2023-12-12T10:09:18.851 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 2, 2023-12-12T10:09:18.851 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2023-12-12T10:09:18.851 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 12102, 2023-12-12T10:09:18.852 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 6051, 2023-12-12T10:09:18.852 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 628, 2023-12-12T10:09:18.852 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2023-12-12T10:09:18.852 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.852 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2023-12-12T10:09:18.852 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:09:18.852 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2023-12-12T10:09:18.852 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:09:18.852 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:09:18.853 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:09:18.853 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:09:18.853 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:09:18.853 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:09:18.853 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:09:18.853 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:09:18.853 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:09:18.853 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:09:18.853 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:09:18.854 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:09:18.854 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:09:18.854 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2023-12-12T10:09:18.854 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:09:18.854 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2023-12-12T10:09:18.854 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:09:18.854 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:09:18.854 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2023-12-12T10:09:18.855 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2023-12-12T10:09:18.856 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2023-12-12T10:09:18.856 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:09:18.856 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:09:18.856 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:09:18.856 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:09:18.856 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 308215, 2023-12-12T10:09:18.856 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 149, 2023-12-12T10:09:18.856 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 1672, 2023-12-12T10:09:18.856 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 19, 2023-12-12T10:09:18.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2023-12-12T10:09:18.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2023-12-12T10:09:18.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:09:18.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2023-12-12T10:09:18.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:09:18.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2023-12-12T10:09:18.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 9848, 2023-12-12T10:09:18.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 190, 2023-12-12T10:09:18.857 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:09:18.858 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:09:18.858 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2023-12-12T10:09:18.858 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2023-12-12T10:09:18.858 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 138752, 2023-12-12T10:09:18.858 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 646, 2023-12-12T10:09:18.858 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2023-12-12T10:09:18.858 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2023-12-12T10:09:18.858 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2023-12-12T10:09:18.858 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2023-12-12T10:09:18.859 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.859 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2023-12-12T10:09:18.859 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2023-12-12T10:09:18.859 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2023-12-12T10:09:18.859 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 26027, 2023-12-12T10:09:18.859 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 125720021, 2023-12-12T10:09:18.859 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 33, 2023-12-12T10:09:18.859 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 26027, 2023-12-12T10:09:18.859 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2023-12-12T10:09:18.860 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 26027, 2023-12-12T10:09:18.860 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 274.082945344, 2023-12-12T10:09:18.860 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.010530716 2023-12-12T10:09:18.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.860 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2023-12-12T10:09:18.860 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2023-12-12T10:09:18.860 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 26027, 2023-12-12T10:09:18.860 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 26488 2023-12-12T10:09:18.860 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.861 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 26027, 2023-12-12T10:09:18.861 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 491, 2023-12-12T10:09:18.861 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 25536, 2023-12-12T10:09:18.861 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2023-12-12T10:09:18.861 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2023-12-12T10:09:18.861 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2023-12-12T10:09:18.861 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2023-12-12T10:09:18.861 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 203, 2023-12-12T10:09:18.861 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 3386, 2023-12-12T10:09:18.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 93, 2023-12-12T10:09:18.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2023-12-12T10:09:18.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2023-12-12T10:09:18.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2023-12-12T10:09:18.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2023-12-12T10:09:18.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 21767, 2023-12-12T10:09:18.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2023-12-12T10:09:18.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2023-12-12T10:09:18.862 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:09:18.863 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2023-12-12T10:09:18.864 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:09:18.864 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2023-12-12T10:09:18.864 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2023-12-12T10:09:18.864 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2023-12-12T10:09:18.864 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2023-12-12T10:09:18.864 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2023-12-12T10:09:18.864 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2023-12-12T10:09:18.864 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2023-12-12T10:09:18.864 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2023-12-12T10:09:18.865 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2023-12-12T10:09:18.865 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2023-12-12T10:09:18.865 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2023-12-12T10:09:18.865 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2023-12-12T10:09:18.865 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2023-12-12T10:09:18.865 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2023-12-12T10:09:18.865 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2023-12-12T10:09:18.865 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2023-12-12T10:09:18.865 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2023-12-12T10:09:18.866 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 99, 2023-12-12T10:09:18.866 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2023-12-12T10:09:18.866 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 12, 2023-12-12T10:09:18.866 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 12, 2023-12-12T10:09:18.866 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 12, 2023-12-12T10:09:18.866 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2023-12-12T10:09:18.866 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2023-12-12T10:09:18.866 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 545, 2023-12-12T10:09:18.866 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 435, 2023-12-12T10:09:18.867 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 29 2023-12-12T10:09:18.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.867 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2023-12-12T10:09:18.867 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2023-12-12T10:09:18.867 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:09:18.867 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2023-12-12T10:09:18.867 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2023-12-12T10:09:18.867 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.867 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2023-12-12T10:09:18.868 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2023-12-12T10:09:18.868 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.868 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2023-12-12T10:09:18.868 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2023-12-12T10:09:18.868 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 10, 2023-12-12T10:09:18.868 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2023-12-12T10:09:18.868 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 5, 2023-12-12T10:09:18.868 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 10849, 2023-12-12T10:09:18.868 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2023-12-12T10:09:18.869 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.869 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:09:18.869 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:18.869 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2023-12-12T10:09:18.869 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:18.869 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 102667, 2023-12-12T10:09:18.869 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 112353519, 2023-12-12T10:09:18.869 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:18.869 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 102667, 2023-12-12T10:09:18.870 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:18.870 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:18.870 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 102667, 2023-12-12T10:09:18.870 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 112353519, 2023-12-12T10:09:18.870 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:18.870 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.870 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.870 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.870 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:18.871 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.871 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2023-12-12T10:09:18.871 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:18.871 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2023-12-12T10:09:18.871 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:18.871 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:18.871 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:18.871 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:18.871 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:18.872 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 26027, 2023-12-12T10:09:18.872 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 127041521, 2023-12-12T10:09:18.872 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 26027, 2023-12-12T10:09:18.872 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 127041521, 2023-12-12T10:09:18.872 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:18.872 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.872 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.872 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.872 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:18.873 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.873 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2023-12-12T10:09:18.873 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:18.873 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2023-12-12T10:09:18.873 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:18.873 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:18.873 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:18.873 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:18.873 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:18.874 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 26027, 2023-12-12T10:09:18.874 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 26027, 2023-12-12T10:09:18.874 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 26027, 2023-12-12T10:09:18.874 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 26027, 2023-12-12T10:09:18.874 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:18.874 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.874 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.874 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.874 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:18.875 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.875 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:09:18.875 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:18.875 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2023-12-12T10:09:18.875 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:18.875 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 10849, 2023-12-12T10:09:18.875 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 1327917, 2023-12-12T10:09:18.875 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:18.875 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 10849, 2023-12-12T10:09:18.876 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:18.876 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:18.876 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 203, 2023-12-12T10:09:18.876 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1327917, 2023-12-12T10:09:18.876 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:18.876 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.876 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.877 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.877 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:18.877 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:18.877 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x55d78d31a6e0": { 2023-12-12T10:09:18.877 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:18.877 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2023-12-12T10:09:18.877 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:18.877 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 47777, 2023-12-12T10:09:18.877 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 121512280, 2023-12-12T10:09:18.878 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:18.878 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 47777, 2023-12-12T10:09:18.878 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:18.878 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:18.878 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 3183, 2023-12-12T10:09:18.878 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 121512280, 2023-12-12T10:09:18.878 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:18.878 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:18.878 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:18.879 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:18.879 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:18.879 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:18.879 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-12-12T10:09:19.185 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:18 smithi173 ceph-mon[103308]: pgmap v330: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:09:19.222 INFO:tasks.check_counter:Daemon mds.f mds.exported=1328 2023-12-12T10:09:19.222 INFO:tasks.check_counter:Daemon mds.f mds.imported=1125 2023-12-12T10:09:19.222 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=738 2023-12-12T10:09:19.222 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=6209 2023-12-12T10:09:19.223 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2023-12-12T10:09:19.223 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-12-12T10:09:19.223 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-12-12T10:09:19.223 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=29 2023-12-12T10:09:19.223 DEBUG:tasks.check_counter:Getting stats from i 2023-12-12T10:09:19.223 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.i perf dump 2023-12-12T10:09:19.250 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:18 smithi111 ceph-mon[102565]: pgmap v330: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:09:19.407 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:18 smithi042 ceph-mon[98894]: pgmap v330: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 717 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 11 op/s 2023-12-12T10:09:20.042 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T10:09:20.248 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:19 smithi111 ceph-mon[102565]: pgmap v331: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 182 B/s wr, 15 op/s 2023-12-12T10:09:20.311 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:19 smithi173 ceph-mon[103308]: pgmap v331: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 182 B/s wr, 15 op/s 2023-12-12T10:09:20.410 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:19 smithi042 ceph-mon[98894]: pgmap v331: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 182 B/s wr, 15 op/s 2023-12-12T10:09:22.404 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:22 smithi042 ceph-mon[98894]: pgmap v332: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:09:22.497 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:22 smithi111 ceph-mon[102565]: pgmap v332: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:09:22.575 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:22 smithi173 ceph-mon[103308]: pgmap v332: 129 pgs: 22 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 37 active+clean; 717 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:09:24.718 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-12-12T10:09:24.718 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:09:24.719 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 96407, 2023-12-12T10:09:24.719 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 149593, 2023-12-12T10:09:24.719 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 82957184, 2023-12-12T10:09:24.719 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 223837349, 2023-12-12T10:09:24.719 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 11, 2023-12-12T10:09:24.719 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 4, 2023-12-12T10:09:24.719 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 5.181362590, 2023-12-12T10:09:24.719 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 2.159787592, 2023-12-12T10:09:24.720 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 111.745318950, 2023-12-12T10:09:24.720 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.326450850, 2023-12-12T10:09:24.720 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:24.720 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 149592, 2023-12-12T10:09:24.720 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 35.098843812, 2023-12-12T10:09:24.720 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000234630 2023-12-12T10:09:24.720 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.720 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:24.720 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 202016, 2023-12-12T10:09:24.721 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.035336080, 2023-12-12T10:09:24.721 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000174 2023-12-12T10:09:24.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.721 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-12T10:09:24.721 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 448 2023-12-12T10:09:24.721 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.721 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:09:24.721 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 58229, 2023-12-12T10:09:24.721 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 53319, 2023-12-12T10:09:24.722 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 134779349, 2023-12-12T10:09:24.722 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 194280188, 2023-12-12T10:09:24.722 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 13, 2023-12-12T10:09:24.722 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 7, 2023-12-12T10:09:24.722 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 3.418148046, 2023-12-12T10:09:24.722 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 1.467807710, 2023-12-12T10:09:24.722 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 2.264840069, 2023-12-12T10:09:24.722 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.457888956, 2023-12-12T10:09:24.722 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:24.723 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 53319, 2023-12-12T10:09:24.723 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 14.908400911, 2023-12-12T10:09:24.723 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000279607 2023-12-12T10:09:24.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.723 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:24.723 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 84104, 2023-12-12T10:09:24.723 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.019823589, 2023-12-12T10:09:24.723 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000235 2023-12-12T10:09:24.723 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 2512, 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 120160 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 12503, 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 12604, 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 7546493, 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 115135108, 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 15, 2023-12-12T10:09:24.724 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 8, 2023-12-12T10:09:24.725 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 1.218145151, 2023-12-12T10:09:24.725 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.433602858, 2023-12-12T10:09:24.725 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.223096123, 2023-12-12T10:09:24.725 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.522592468, 2023-12-12T10:09:24.725 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:24.725 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 12600, 2023-12-12T10:09:24.725 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2.701917535, 2023-12-12T10:09:24.725 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000214437 2023-12-12T10:09:24.725 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.726 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:24.726 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 22, 2023-12-12T10:09:24.726 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000003047, 2023-12-12T10:09:24.726 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000138 2023-12-12T10:09:24.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.726 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1631920, 2023-12-12T10:09:24.726 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 453440 2023-12-12T10:09:24.726 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.726 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 12421, 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 56.379773732, 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.004539068 2023-12-12T10:09:24.727 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:24.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.728 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2023-12-12T10:09:24.728 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2023-12-12T10:09:24.728 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2023-12-12T10:09:24.728 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 15466, 2023-12-12T10:09:24.728 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 7.840122049, 2023-12-12T10:09:24.728 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000506926 2023-12-12T10:09:24.728 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:24.728 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.729 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2023-12-12T10:09:24.729 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 30804, 2023-12-12T10:09:24.729 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 22117, 2023-12-12T10:09:24.729 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2023-12-12T10:09:24.729 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 22117, 2023-12-12T10:09:24.729 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 119.710060710, 2023-12-12T10:09:24.729 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.005412581 2023-12-12T10:09:24.729 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.729 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2023-12-12T10:09:24.730 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 8687, 2023-12-12T10:09:24.730 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 347, 2023-12-12T10:09:24.730 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2023-12-12T10:09:24.730 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 235, 2023-12-12T10:09:24.730 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 28, 2023-12-12T10:09:24.730 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2023-12-12T10:09:24.730 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 22570, 2023-12-12T10:09:24.730 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 2, 2023-12-12T10:09:24.730 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 6, 2023-12-12T10:09:24.731 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 22562, 2023-12-12T10:09:24.731 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 22567, 2023-12-12T10:09:24.731 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 196389, 2023-12-12T10:09:24.731 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2023-12-12T10:09:24.731 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2023-12-12T10:09:24.731 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 3, 2023-12-12T10:09:24.731 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 40839, 2023-12-12T10:09:24.731 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 31624, 2023-12-12T10:09:24.731 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 8279, 2023-12-12T10:09:24.732 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 11, 2023-12-12T10:09:24.732 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 63, 2023-12-12T10:09:24.732 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2023-12-12T10:09:24.732 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2023-12-12T10:09:24.732 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2023-12-12T10:09:24.732 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2023-12-12T10:09:24.732 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 3831, 2023-12-12T10:09:24.732 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 199865, 2023-12-12T10:09:24.732 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 4631, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 207854, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 4, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 246, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 6, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2023-12-12T10:09:24.733 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:09:24.734 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:09:24.734 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:09:24.734 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:09:24.734 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:09:24.734 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2023-12-12T10:09:24.734 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 28, 2023-12-12T10:09:24.734 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 24, 2023-12-12T10:09:24.734 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 11789, 2023-12-12T10:09:24.734 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2023-12-12T10:09:24.735 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-12T10:09:24.735 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:09:24.735 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 17973, 2023-12-12T10:09:24.735 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 17948, 2023-12-12T10:09:24.735 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 269, 2023-12-12T10:09:24.735 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 19196 2023-12-12T10:09:24.735 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.735 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2023-12-12T10:09:24.735 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 2138, 2023-12-12T10:09:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 1162, 2023-12-12T10:09:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 584, 2023-12-12T10:09:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 3363, 2023-12-12T10:09:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 6181, 2023-12-12T10:09:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 1, 2023-12-12T10:09:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2023-12-12T10:09:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:09:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 8739, 2023-12-12T10:09:24.736 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 8738, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 5688, 2023-12-12T10:09:24.737 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2023-12-12T10:09:24.738 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 28, 2023-12-12T10:09:24.738 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2023-12-12T10:09:24.738 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2023-12-12T10:09:24.738 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.738 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2023-12-12T10:09:24.738 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 5, 2023-12-12T10:09:24.738 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 54088, 2023-12-12T10:09:24.738 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2023-12-12T10:09:24.738 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2023-12-12T10:09:24.739 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 54088, 2023-12-12T10:09:24.739 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2023-12-12T10:09:24.739 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2023-12-12T10:09:24.739 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 95, 2023-12-12T10:09:24.739 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2023-12-12T10:09:24.739 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2023-12-12T10:09:24.739 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 95, 2023-12-12T10:09:24.739 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 6, 2023-12-12T10:09:24.739 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 280681142, 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 0, 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 9233, 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 425.890093855, 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.046126946 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.740 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 0 2023-12-12T10:09:24.741 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.741 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2023-12-12T10:09:24.741 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 12574, 2023-12-12T10:09:24.741 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 134632, 2023-12-12T10:09:24.741 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 122058, 2023-12-12T10:09:24.741 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 532, 2023-12-12T10:09:24.741 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 2667, 2023-12-12T10:09:24.741 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 2135, 2023-12-12T10:09:24.741 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 22570, 2023-12-12T10:09:24.742 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 227697, 2023-12-12T10:09:24.742 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 205127, 2023-12-12T10:09:24.742 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2023-12-12T10:09:24.742 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 21125, 2023-12-12T10:09:24.742 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 21125, 2023-12-12T10:09:24.742 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 285148, 2023-12-12T10:09:24.742 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 199036 2023-12-12T10:09:24.742 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.742 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2023-12-12T10:09:24.743 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 34735, 2023-12-12T10:09:24.743 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 1795, 2023-12-12T10:09:24.743 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 30804, 2023-12-12T10:09:24.743 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 269, 2023-12-12T10:09:24.743 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 3887, 2023-12-12T10:09:24.743 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2023-12-12T10:09:24.743 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2867, 2023-12-12T10:09:24.743 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2.777668964, 2023-12-12T10:09:24.743 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000968841 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 7, 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.046540882, 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.006648697 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.744 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.745 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2023-12-12T10:09:24.745 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.745 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.745 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.745 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.745 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2023-12-12T10:09:24.745 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 8514, 2023-12-12T10:09:24.745 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 29.610895016, 2023-12-12T10:09:24.746 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.003477906 2023-12-12T10:09:24.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.746 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2023-12-12T10:09:24.746 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.746 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.746 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.746 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.746 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2023-12-12T10:09:24.746 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.747 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.747 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.747 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.747 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2023-12-12T10:09:24.747 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.747 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.747 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2023-12-12T10:09:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2023-12-12T10:09:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.748 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2023-12-12T10:09:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2023-12-12T10:09:24.749 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2023-12-12T10:09:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.750 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.751 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2023-12-12T10:09:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1990, 2023-12-12T10:09:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 31.483670688, 2023-12-12T10:09:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.015820940 2023-12-12T10:09:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2023-12-12T10:09:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.752 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2023-12-12T10:09:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2023-12-12T10:09:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.753 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2023-12-12T10:09:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1, 2023-12-12T10:09:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.092310022, 2023-12-12T10:09:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.092310022 2023-12-12T10:09:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2023-12-12T10:09:24.754 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2023-12-12T10:09:24.755 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2023-12-12T10:09:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.756 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2023-12-12T10:09:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2023-12-12T10:09:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.757 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2023-12-12T10:09:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2023-12-12T10:09:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.758 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2023-12-12T10:09:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 8738, 2023-12-12T10:09:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 55.698975138, 2023-12-12T10:09:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.006374339 2023-12-12T10:09:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:09:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:09:24.759 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2023-12-12T10:09:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2023-12-12T10:09:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 1, 2023-12-12T10:09:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 4, 2023-12-12T10:09:24.760 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 3, 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 1, 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 6954, 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 6954, 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 571, 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:09:24.761 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2023-12-12T10:09:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:09:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:09:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:09:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:09:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:09:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:09:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:09:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:09:24.762 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:09:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:09:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:09:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:09:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:09:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2023-12-12T10:09:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:09:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2023-12-12T10:09:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:09:24.763 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:09:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:09:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:09:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:09:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:09:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:09:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2023-12-12T10:09:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:09:24.764 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 731739, 2023-12-12T10:09:24.765 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 12675, 2023-12-12T10:09:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 264, 2023-12-12T10:09:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 3, 2023-12-12T10:09:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2023-12-12T10:09:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2023-12-12T10:09:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:09:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2023-12-12T10:09:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:09:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2023-12-12T10:09:24.766 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 9848, 2023-12-12T10:09:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 190, 2023-12-12T10:09:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:09:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:09:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2023-12-12T10:09:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2023-12-12T10:09:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 45239506, 2023-12-12T10:09:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 415590, 2023-12-12T10:09:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2023-12-12T10:09:24.767 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2023-12-12T10:09:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2023-12-12T10:09:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2023-12-12T10:09:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2023-12-12T10:09:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2023-12-12T10:09:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2023-12-12T10:09:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 28509, 2023-12-12T10:09:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 280898013, 2023-12-12T10:09:24.768 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 33, 2023-12-12T10:09:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 28509, 2023-12-12T10:09:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2023-12-12T10:09:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 28509, 2023-12-12T10:09:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 328.860735094, 2023-12-12T10:09:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.011535330 2023-12-12T10:09:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2023-12-12T10:09:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2023-12-12T10:09:24.769 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 28509, 2023-12-12T10:09:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 29168 2023-12-12T10:09:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 28509, 2023-12-12T10:09:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 1065, 2023-12-12T10:09:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 27444, 2023-12-12T10:09:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2023-12-12T10:09:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2023-12-12T10:09:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 0, 2023-12-12T10:09:24.770 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 10, 2023-12-12T10:09:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 233, 2023-12-12T10:09:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 9466, 2023-12-12T10:09:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 106, 2023-12-12T10:09:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2023-12-12T10:09:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2023-12-12T10:09:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2023-12-12T10:09:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2023-12-12T10:09:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 17581, 2023-12-12T10:09:24.771 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 0, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 20, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2023-12-12T10:09:24.772 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2023-12-12T10:09:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:09:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2023-12-12T10:09:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:09:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2023-12-12T10:09:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2023-12-12T10:09:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2023-12-12T10:09:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2023-12-12T10:09:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2023-12-12T10:09:24.773 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2023-12-12T10:09:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2023-12-12T10:09:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2023-12-12T10:09:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2023-12-12T10:09:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2023-12-12T10:09:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2023-12-12T10:09:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2023-12-12T10:09:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2023-12-12T10:09:24.774 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2023-12-12T10:09:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2023-12-12T10:09:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2023-12-12T10:09:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2023-12-12T10:09:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 99, 2023-12-12T10:09:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2023-12-12T10:09:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 12, 2023-12-12T10:09:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 12, 2023-12-12T10:09:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 12, 2023-12-12T10:09:24.775 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2023-12-12T10:09:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2023-12-12T10:09:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 542, 2023-12-12T10:09:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 1179, 2023-12-12T10:09:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 31 2023-12-12T10:09:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2023-12-12T10:09:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 1, 2023-12-12T10:09:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:09:24.776 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2023-12-12T10:09:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2023-12-12T10:09:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2023-12-12T10:09:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2023-12-12T10:09:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2023-12-12T10:09:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2023-12-12T10:09:24.777 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 10, 2023-12-12T10:09:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2023-12-12T10:09:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 5, 2023-12-12T10:09:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 8738, 2023-12-12T10:09:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2023-12-12T10:09:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:09:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2023-12-12T10:09:24.778 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 167135, 2023-12-12T10:09:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 210222176, 2023-12-12T10:09:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 167135, 2023-12-12T10:09:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 167135, 2023-12-12T10:09:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 210222176, 2023-12-12T10:09:24.779 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2023-12-12T10:09:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2023-12-12T10:09:24.780 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 28509, 2023-12-12T10:09:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 281933467, 2023-12-12T10:09:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 28509, 2023-12-12T10:09:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 281933467, 2023-12-12T10:09:24.781 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2023-12-12T10:09:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2023-12-12T10:09:24.782 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 28509, 2023-12-12T10:09:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 28509, 2023-12-12T10:09:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 28509, 2023-12-12T10:09:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 28509, 2023-12-12T10:09:24.783 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:09:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2023-12-12T10:09:24.784 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 8738, 2023-12-12T10:09:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 1054970, 2023-12-12T10:09:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 8738, 2023-12-12T10:09:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 233, 2023-12-12T10:09:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 1054970, 2023-12-12T10:09:24.785 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x555a8aba66e0": { 2023-12-12T10:09:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2023-12-12T10:09:24.786 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 54088, 2023-12-12T10:09:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 276486838, 2023-12-12T10:09:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 54088, 2023-12-12T10:09:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 9233, 2023-12-12T10:09:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 276486838, 2023-12-12T10:09:24.787 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:24.788 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:24.788 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-12-12T10:09:24.789 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:24 smithi173 ceph-mon[103308]: pgmap v333: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 717 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:09:24.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:24 smithi042 ceph-mon[98894]: pgmap v333: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 717 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:09:24.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:24 smithi111 ceph-mon[102565]: pgmap v333: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 717 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 9 op/s 2023-12-12T10:09:25.164 INFO:tasks.check_counter:Daemon mds.i mds.exported=3831 2023-12-12T10:09:25.165 INFO:tasks.check_counter:Daemon mds.i mds.imported=4631 2023-12-12T10:09:25.165 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=2138 2023-12-12T10:09:25.165 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1162 2023-12-12T10:09:25.165 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-12-12T10:09:25.165 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-12-12T10:09:25.165 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-12-12T10:09:25.165 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=28 2023-12-12T10:09:25.165 DEBUG:tasks.check_counter:Getting stats from l 2023-12-12T10:09:25.166 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 shell --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 -- ceph daemon mds.l perf dump 2023-12-12T10:09:25.935 INFO:teuthology.orchestra.run.smithi173.stderr:Inferring config /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/mon.c/config 2023-12-12T10:09:26.515 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:26 smithi173 ceph-mon[103308]: pgmap v334: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2023-12-12T10:09:26.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:26 smithi042 ceph-mon[98894]: pgmap v334: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2023-12-12T10:09:26.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:26 smithi111 ceph-mon[102565]: pgmap v334: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 85 B/s wr, 13 op/s 2023-12-12T10:09:27.486 INFO:teuthology.orchestra.run.smithi173.stdout:{ 2023-12-12T10:09:27.487 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-0": { 2023-12-12T10:09:27.487 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 1165, 2023-12-12T10:09:27.487 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1129, 2023-12-12T10:09:27.487 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 60621232, 2023-12-12T10:09:27.487 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 410560, 2023-12-12T10:09:27.487 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 7, 2023-12-12T10:09:27.487 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 3, 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.167858247, 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.038553267, 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.088824339, 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.040243869, 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1129, 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.092734460, 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000082138 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.488 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:27.489 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.489 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.489 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.489 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 1606976, 2023-12-12T10:09:27.489 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 131904 2023-12-12T10:09:27.489 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.489 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-1": { 2023-12-12T10:09:27.489 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 266, 2023-12-12T10:09:27.490 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 428, 2023-12-12T10:09:27.490 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 86706089, 2023-12-12T10:09:27.490 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 515670, 2023-12-12T10:09:27.490 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 9, 2023-12-12T10:09:27.490 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 7, 2023-12-12T10:09:27.493 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.078324790, 2023-12-12T10:09:27.493 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.010241243, 2023-12-12T10:09:27.493 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.056848378, 2023-12-12T10:09:27.494 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.007191980, 2023-12-12T10:09:27.494 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:27.494 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 425, 2023-12-12T10:09:27.494 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.041225853, 2023-12-12T10:09:27.494 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000097002 2023-12-12T10:09:27.494 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.494 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:27.494 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.495 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.495 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.495 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-12T10:09:27.495 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 433504 2023-12-12T10:09:27.495 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.495 INFO:teuthology.orchestra.run.smithi173.stdout: "AsyncMessenger::Worker-2": { 2023-12-12T10:09:27.496 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_messages": 1508, 2023-12-12T10:09:27.496 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages": 1509, 2023-12-12T10:09:27.496 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_bytes": 135725934, 2023-12-12T10:09:27.496 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_bytes": 468684, 2023-12-12T10:09:27.496 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_created_connections": 6, 2023-12-12T10:09:27.496 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_active_connections": 5, 2023-12-12T10:09:27.496 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_total_time": 0.178760176, 2023-12-12T10:09:27.496 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_send_time": 0.032758361, 2023-12-12T10:09:27.496 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_recv_time": 0.095288765, 2023-12-12T10:09:27.497 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_running_fast_dispatch_time": 0.041947137, 2023-12-12T10:09:27.497 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_messages_queue_lat": { 2023-12-12T10:09:27.497 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 1507, 2023-12-12T10:09:27.497 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.088434717, 2023-12-12T10:09:27.497 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000058682 2023-12-12T10:09:27.497 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.497 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_handle_ack_lat": { 2023-12-12T10:09:27.497 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.498 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.499 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.499 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.499 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-12T10:09:27.499 INFO:teuthology.orchestra.run.smithi173.stdout: "msgr_send_encrypted_bytes": 0 2023-12-12T10:09:27.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.500 INFO:teuthology.orchestra.run.smithi173.stdout: "cct": { 2023-12-12T10:09:27.500 INFO:teuthology.orchestra.run.smithi173.stdout: "total_workers": 1, 2023-12-12T10:09:27.500 INFO:teuthology.orchestra.run.smithi173.stdout: "unhealthy_workers": 0 2023-12-12T10:09:27.500 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.500 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-MDSRank": { 2023-12-12T10:09:27.500 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2023-12-12T10:09:27.500 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2023-12-12T10:09:27.500 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 4235, 2023-12-12T10:09:27.501 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.121347284, 2023-12-12T10:09:27.501 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000028653 2023-12-12T10:09:27.501 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:27.501 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.501 INFO:teuthology.orchestra.run.smithi173.stdout: "finisher-PurgeQueue": { 2023-12-12T10:09:27.501 INFO:teuthology.orchestra.run.smithi173.stdout: "queue_len": 0, 2023-12-12T10:09:27.501 INFO:teuthology.orchestra.run.smithi173.stdout: "complete_latency": { 2023-12-12T10:09:27.501 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.501 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.502 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.502 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:27.502 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.502 INFO:teuthology.orchestra.run.smithi173.stdout: "mds": { 2023-12-12T10:09:27.502 INFO:teuthology.orchestra.run.smithi173.stdout: "request": 0, 2023-12-12T10:09:27.502 INFO:teuthology.orchestra.run.smithi173.stdout: "reply": 0, 2023-12-12T10:09:27.502 INFO:teuthology.orchestra.run.smithi173.stdout: "reply_latency": { 2023-12-12T10:09:27.502 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.503 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.503 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.503 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.503 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_reply": 0, 2023-12-12T10:09:27.503 INFO:teuthology.orchestra.run.smithi173.stdout: "forward": 0, 2023-12-12T10:09:27.503 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_complete": 0, 2023-12-12T10:09:27.503 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_fetch_keys": 0, 2023-12-12T10:09:27.503 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_commit": 0, 2023-12-12T10:09:27.503 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_split": 0, 2023-12-12T10:09:27.504 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_merge": 0, 2023-12-12T10:09:27.504 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes": 31301, 2023-12-12T10:09:27.504 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_top": 0, 2023-12-12T10:09:27.504 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_bottom": 31301, 2023-12-12T10:09:27.504 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pin_tail": 0, 2023-12-12T10:09:27.504 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_pinned": 31300, 2023-12-12T10:09:27.504 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_expired": 0, 2023-12-12T10:09:27.504 INFO:teuthology.orchestra.run.smithi173.stdout: "inodes_with_caps": 0, 2023-12-12T10:09:27.504 INFO:teuthology.orchestra.run.smithi173.stdout: "caps": 0, 2023-12-12T10:09:27.505 INFO:teuthology.orchestra.run.smithi173.stdout: "subtrees": 1, 2023-12-12T10:09:27.505 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse": 0, 2023-12-12T10:09:27.505 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_hit": 0, 2023-12-12T10:09:27.505 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_forward": 0, 2023-12-12T10:09:27.505 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_discover": 0, 2023-12-12T10:09:27.505 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_dir_fetch": 0, 2023-12-12T10:09:27.505 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_remote_ino": 0, 2023-12-12T10:09:27.505 INFO:teuthology.orchestra.run.smithi173.stdout: "traverse_lock": 0, 2023-12-12T10:09:27.505 INFO:teuthology.orchestra.run.smithi173.stdout: "load_cent": 0, 2023-12-12T10:09:27.506 INFO:teuthology.orchestra.run.smithi173.stdout: "q": 0, 2023-12-12T10:09:27.506 INFO:teuthology.orchestra.run.smithi173.stdout: "exported": 0, 2023-12-12T10:09:27.506 INFO:teuthology.orchestra.run.smithi173.stdout: "exported_inodes": 0, 2023-12-12T10:09:27.506 INFO:teuthology.orchestra.run.smithi173.stdout: "imported": 0, 2023-12-12T10:09:27.506 INFO:teuthology.orchestra.run.smithi173.stdout: "imported_inodes": 0, 2023-12-12T10:09:27.506 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_dir_fetch": 0, 2023-12-12T10:09:27.506 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_backtrace_fetch": 0, 2023-12-12T10:09:27.506 INFO:teuthology.orchestra.run.smithi173.stdout: "openino_peer_discover": 0, 2023-12-12T10:09:27.506 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rfiles": 0, 2023-12-12T10:09:27.507 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rbytes": 0, 2023-12-12T10:09:27.507 INFO:teuthology.orchestra.run.smithi173.stdout: "root_rsnaps": 0, 2023-12-12T10:09:27.507 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_fetch": 0, 2023-12-12T10:09:27.507 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_set_tag": 0, 2023-12-12T10:09:27.507 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_backtrace_repaired": 0, 2023-12-12T10:09:27.507 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_inotable_repaired": 0, 2023-12-12T10:09:27.507 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_inodes": 0, 2023-12-12T10:09:27.507 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dir_base_inodes": 0, 2023-12-12T10:09:27.507 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_dirfrag_rstats": 0, 2023-12-12T10:09:27.508 INFO:teuthology.orchestra.run.smithi173.stdout: "scrub_file_inodes": 0, 2023-12-12T10:09:27.508 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_inode_file_caps": 0, 2023-12-12T10:09:27.508 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_revoke": 0, 2023-12-12T10:09:27.508 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_grant": 0, 2023-12-12T10:09:27.508 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_trunc": 0, 2023-12-12T10:09:27.508 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-12T10:09:27.508 INFO:teuthology.orchestra.run.smithi173.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-12T10:09:27.508 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps": 0, 2023-12-12T10:09:27.508 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_caps_dirty": 0, 2023-12-12T10:09:27.509 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_cap_release": 0, 2023-12-12T10:09:27.509 INFO:teuthology.orchestra.run.smithi173.stdout: "process_request_cap_release": 0 2023-12-12T10:09:27.509 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.509 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_cache": { 2023-12-12T10:09:27.509 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update": 0, 2023-12-12T10:09:27.509 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_update_receipt": 0, 2023-12-12T10:09:27.509 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_try_discover": 0, 2023-12-12T10:09:27.509 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_send_discover": 0, 2023-12-12T10:09:27.509 INFO:teuthology.orchestra.run.smithi173.stdout: "dir_handle_discover": 0, 2023-12-12T10:09:27.510 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays": 0, 2023-12-12T10:09:27.510 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_delayed": 0, 2023-12-12T10:09:27.510 INFO:teuthology.orchestra.run.smithi173.stdout: "num_strays_enqueuing": 0, 2023-12-12T10:09:27.510 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_created": 0, 2023-12-12T10:09:27.510 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_enqueued": 0, 2023-12-12T10:09:27.510 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_reintegrated": 0, 2023-12-12T10:09:27.510 INFO:teuthology.orchestra.run.smithi173.stdout: "strays_migrated": 0, 2023-12-12T10:09:27.510 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_processing": 0, 2023-12-12T10:09:27.510 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_enqueued": 0, 2023-12-12T10:09:27.511 INFO:teuthology.orchestra.run.smithi173.stdout: "num_recovering_prioritized": 0, 2023-12-12T10:09:27.511 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_started": 0, 2023-12-12T10:09:27.511 INFO:teuthology.orchestra.run.smithi173.stdout: "recovery_completed": 0, 2023-12-12T10:09:27.511 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_enqueue_scrub": 0, 2023-12-12T10:09:27.511 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_exportdir": 0, 2023-12-12T10:09:27.511 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_flush": 0, 2023-12-12T10:09:27.511 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragmentdir": 0, 2023-12-12T10:09:27.511 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_fragstats": 0, 2023-12-12T10:09:27.511 INFO:teuthology.orchestra.run.smithi173.stdout: "ireq_inodestats": 0 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_log": { 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: "evlrg": 0, 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: "evadd": 0, 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: "evex": 0, 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: "evtrm": 0, 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: "ev": 54088, 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: "evexg": 0, 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: "evexd": 0, 2023-12-12T10:09:27.512 INFO:teuthology.orchestra.run.smithi173.stdout: "segadd": 0, 2023-12-12T10:09:27.513 INFO:teuthology.orchestra.run.smithi173.stdout: "segex": 0, 2023-12-12T10:09:27.513 INFO:teuthology.orchestra.run.smithi173.stdout: "segtrm": 0, 2023-12-12T10:09:27.513 INFO:teuthology.orchestra.run.smithi173.stdout: "seg": 95, 2023-12-12T10:09:27.513 INFO:teuthology.orchestra.run.smithi173.stdout: "segmjr": 6, 2023-12-12T10:09:27.513 INFO:teuthology.orchestra.run.smithi173.stdout: "segexg": 0, 2023-12-12T10:09:27.513 INFO:teuthology.orchestra.run.smithi173.stdout: "segexd": 0, 2023-12-12T10:09:27.513 INFO:teuthology.orchestra.run.smithi173.stdout: "expos": 4194304, 2023-12-12T10:09:27.513 INFO:teuthology.orchestra.run.smithi173.stdout: "wrpos": 280681142, 2023-12-12T10:09:27.513 INFO:teuthology.orchestra.run.smithi173.stdout: "rdpos": 280501365, 2023-12-12T10:09:27.514 INFO:teuthology.orchestra.run.smithi173.stdout: "jlat": { 2023-12-12T10:09:27.514 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.514 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.514 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.514 INFO:teuthology.orchestra.run.smithi173.stdout: "replayed": 54088 2023-12-12T10:09:27.514 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.515 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_mem": { 2023-12-12T10:09:27.515 INFO:teuthology.orchestra.run.smithi173.stdout: "ino": 12564, 2023-12-12T10:09:27.515 INFO:teuthology.orchestra.run.smithi173.stdout: "ino+": 134083, 2023-12-12T10:09:27.515 INFO:teuthology.orchestra.run.smithi173.stdout: "ino-": 121519, 2023-12-12T10:09:27.515 INFO:teuthology.orchestra.run.smithi173.stdout: "dir": 523, 2023-12-12T10:09:27.515 INFO:teuthology.orchestra.run.smithi173.stdout: "dir+": 2199, 2023-12-12T10:09:27.515 INFO:teuthology.orchestra.run.smithi173.stdout: "dir-": 1676, 2023-12-12T10:09:27.515 INFO:teuthology.orchestra.run.smithi173.stdout: "dn": 31301, 2023-12-12T10:09:27.516 INFO:teuthology.orchestra.run.smithi173.stdout: "dn+": 231574, 2023-12-12T10:09:27.516 INFO:teuthology.orchestra.run.smithi173.stdout: "dn-": 200273, 2023-12-12T10:09:27.516 INFO:teuthology.orchestra.run.smithi173.stdout: "cap": 0, 2023-12-12T10:09:27.516 INFO:teuthology.orchestra.run.smithi173.stdout: "cap+": 0, 2023-12-12T10:09:27.516 INFO:teuthology.orchestra.run.smithi173.stdout: "cap-": 0, 2023-12-12T10:09:27.516 INFO:teuthology.orchestra.run.smithi173.stdout: "rss": 183796, 2023-12-12T10:09:27.517 INFO:teuthology.orchestra.run.smithi173.stdout: "heap": 207228 2023-12-12T10:09:27.517 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.517 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_server": { 2023-12-12T10:09:27.517 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_client_request": 0, 2023-12-12T10:09:27.517 INFO:teuthology.orchestra.run.smithi173.stdout: "dispatch_server_request": 0, 2023-12-12T10:09:27.517 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_request": 0, 2023-12-12T10:09:27.517 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_client_session": 0, 2023-12-12T10:09:27.517 INFO:teuthology.orchestra.run.smithi173.stdout: "handle_peer_request": 0, 2023-12-12T10:09:27.517 INFO:teuthology.orchestra.run.smithi173.stdout: "req_create_latency": { 2023-12-12T10:09:27.518 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.518 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.518 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.518 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getattr_latency": { 2023-12-12T10:09:27.518 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.518 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.518 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.518 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.519 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getfilelock_latency": { 2023-12-12T10:09:27.519 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.519 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.519 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.519 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.519 INFO:teuthology.orchestra.run.smithi173.stdout: "req_link_latency": { 2023-12-12T10:09:27.519 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.519 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.519 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.520 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookup_latency": { 2023-12-12T10:09:27.520 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.520 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.520 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.520 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.520 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookuphash_latency": { 2023-12-12T10:09:27.520 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.520 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.521 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.521 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupino_latency": { 2023-12-12T10:09:27.521 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.521 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.521 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.521 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.521 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupname_latency": { 2023-12-12T10:09:27.521 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupparent_latency": { 2023-12-12T10:09:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.522 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lookupsnap_latency": { 2023-12-12T10:09:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "req_lssnap_latency": { 2023-12-12T10:09:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.523 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mkdir_latency": { 2023-12-12T10:09:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mknod_latency": { 2023-12-12T10:09:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.524 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "req_mksnap_latency": { 2023-12-12T10:09:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "req_open_latency": { 2023-12-12T10:09:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.525 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "req_readdir_latency": { 2023-12-12T10:09:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rename_latency": { 2023-12-12T10:09:27.526 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "req_renamesnap_latency": { 2023-12-12T10:09:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.527 INFO:teuthology.orchestra.run.smithi173.stdout: "req_snapdiff_latency": { 2023-12-12T10:09:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmdir_latency": { 2023-12-12T10:09:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.528 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmsnap_latency": { 2023-12-12T10:09:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "req_rmxattr_latency": { 2023-12-12T10:09:27.529 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setattr_latency": { 2023-12-12T10:09:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.530 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setdirlayout_latency": { 2023-12-12T10:09:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setfilelock_latency": { 2023-12-12T10:09:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.531 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.532 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setlayout_latency": { 2023-12-12T10:09:27.532 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.532 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.532 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.532 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.532 INFO:teuthology.orchestra.run.smithi173.stdout: "req_setxattr_latency": { 2023-12-12T10:09:27.532 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.532 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.533 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.533 INFO:teuthology.orchestra.run.smithi173.stdout: "req_symlink_latency": { 2023-12-12T10:09:27.533 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.533 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.533 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.533 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.533 INFO:teuthology.orchestra.run.smithi173.stdout: "req_unlink_latency": { 2023-12-12T10:09:27.533 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.534 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.534 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.534 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.534 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_revoke_eviction": 0, 2023-12-12T10:09:27.534 INFO:teuthology.orchestra.run.smithi173.stdout: "cap_acquisition_throttle": 0, 2023-12-12T10:09:27.534 INFO:teuthology.orchestra.run.smithi173.stdout: "req_getvxattr_latency": { 2023-12-12T10:09:27.534 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.534 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.535 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.535 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:27.535 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.535 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_sessions": { 2023-12-12T10:09:27.535 INFO:teuthology.orchestra.run.smithi173.stdout: "session_count": 1, 2023-12-12T10:09:27.535 INFO:teuthology.orchestra.run.smithi173.stdout: "session_add": 4, 2023-12-12T10:09:27.535 INFO:teuthology.orchestra.run.smithi173.stdout: "session_remove": 3, 2023-12-12T10:09:27.535 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_open": 3, 2023-12-12T10:09:27.535 INFO:teuthology.orchestra.run.smithi173.stdout: "sessions_stale": 0, 2023-12-12T10:09:27.536 INFO:teuthology.orchestra.run.smithi173.stdout: "total_load": 0, 2023-12-12T10:09:27.536 INFO:teuthology.orchestra.run.smithi173.stdout: "average_load": 0, 2023-12-12T10:09:27.536 INFO:teuthology.orchestra.run.smithi173.stdout: "avg_session_uptime": 5088, 2023-12-12T10:09:27.536 INFO:teuthology.orchestra.run.smithi173.stdout: "mdthresh_evicted": 0 2023-12-12T10:09:27.536 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.536 INFO:teuthology.orchestra.run.smithi173.stdout: "mempool": { 2023-12-12T10:09:27.536 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_bytes": 0, 2023-12-12T10:09:27.536 INFO:teuthology.orchestra.run.smithi173.stdout: "bloom_filter_items": 0, 2023-12-12T10:09:27.536 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_bytes": 0, 2023-12-12T10:09:27.537 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_alloc_items": 0, 2023-12-12T10:09:27.537 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_bytes": 0, 2023-12-12T10:09:27.537 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_data_items": 0, 2023-12-12T10:09:27.537 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-12T10:09:27.537 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_onode_items": 0, 2023-12-12T10:09:27.537 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-12T10:09:27.537 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_meta_items": 0, 2023-12-12T10:09:27.537 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_bytes": 0, 2023-12-12T10:09:27.537 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_other_items": 0, 2023-12-12T10:09:27.538 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-12T10:09:27.538 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_cache_buffer_items": 0, 2023-12-12T10:09:27.538 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_bytes": 0, 2023-12-12T10:09:27.538 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_extent_items": 0, 2023-12-12T10:09:27.538 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_bytes": 0, 2023-12-12T10:09:27.538 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_blob_items": 0, 2023-12-12T10:09:27.538 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-12T10:09:27.538 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_shared_blob_items": 0, 2023-12-12T10:09:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-12T10:09:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_inline_bl_items": 0, 2023-12-12T10:09:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_bytes": 0, 2023-12-12T10:09:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_fsck_items": 0, 2023-12-12T10:09:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_bytes": 0, 2023-12-12T10:09:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_txc_items": 0, 2023-12-12T10:09:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-12T10:09:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_deferred_items": 0, 2023-12-12T10:09:27.539 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_bytes": 0, 2023-12-12T10:09:27.540 INFO:teuthology.orchestra.run.smithi173.stdout: "bluestore_writing_items": 0, 2023-12-12T10:09:27.540 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_bytes": 0, 2023-12-12T10:09:27.540 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_items": 0, 2023-12-12T10:09:27.540 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_bytes": 0, 2023-12-12T10:09:27.540 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_reader_items": 0, 2023-12-12T10:09:27.540 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_bytes": 0, 2023-12-12T10:09:27.540 INFO:teuthology.orchestra.run.smithi173.stdout: "bluefs_file_writer_items": 0, 2023-12-12T10:09:27.540 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_bytes": 666381, 2023-12-12T10:09:27.540 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_anon_items": 12635, 2023-12-12T10:09:27.541 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_bytes": 0, 2023-12-12T10:09:27.541 INFO:teuthology.orchestra.run.smithi173.stdout: "buffer_meta_items": 0, 2023-12-12T10:09:27.541 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_bytes": 0, 2023-12-12T10:09:27.541 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_items": 0, 2023-12-12T10:09:27.541 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_bytes": 0, 2023-12-12T10:09:27.541 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_mapbl_items": 0, 2023-12-12T10:09:27.541 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_bytes": 0, 2023-12-12T10:09:27.541 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_pglog_items": 0, 2023-12-12T10:09:27.541 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_bytes": 9848, 2023-12-12T10:09:27.542 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_items": 190, 2023-12-12T10:09:27.542 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_bytes": 0, 2023-12-12T10:09:27.542 INFO:teuthology.orchestra.run.smithi173.stdout: "osdmap_mapping_items": 0, 2023-12-12T10:09:27.542 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_bytes": 0, 2023-12-12T10:09:27.542 INFO:teuthology.orchestra.run.smithi173.stdout: "pgmap_items": 0, 2023-12-12T10:09:27.542 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_bytes": 50978166, 2023-12-12T10:09:27.542 INFO:teuthology.orchestra.run.smithi173.stdout: "mds_co_items": 407278, 2023-12-12T10:09:27.542 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_bytes": 0, 2023-12-12T10:09:27.542 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_1_items": 0, 2023-12-12T10:09:27.543 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_bytes": 0, 2023-12-12T10:09:27.543 INFO:teuthology.orchestra.run.smithi173.stdout: "unittest_2_items": 0 2023-12-12T10:09:27.543 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.543 INFO:teuthology.orchestra.run.smithi173.stdout: "objecter": { 2023-12-12T10:09:27.543 INFO:teuthology.orchestra.run.smithi173.stdout: "op_active": 0, 2023-12-12T10:09:27.543 INFO:teuthology.orchestra.run.smithi173.stdout: "op_laggy": 0, 2023-12-12T10:09:27.543 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send": 2667, 2023-12-12T10:09:27.543 INFO:teuthology.orchestra.run.smithi173.stdout: "op_send_bytes": 21, 2023-12-12T10:09:27.543 INFO:teuthology.orchestra.run.smithi173.stdout: "op_resend": 3, 2023-12-12T10:09:27.544 INFO:teuthology.orchestra.run.smithi173.stdout: "op_reply": 2667, 2023-12-12T10:09:27.544 INFO:teuthology.orchestra.run.smithi173.stdout: "op_latency": { 2023-12-12T10:09:27.544 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2667, 2023-12-12T10:09:27.544 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 4.539900292, 2023-12-12T10:09:27.544 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.001702249 2023-12-12T10:09:27.544 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.544 INFO:teuthology.orchestra.run.smithi173.stdout: "op_inflight": 0, 2023-12-12T10:09:27.544 INFO:teuthology.orchestra.run.smithi173.stdout: "oplen_avg": { 2023-12-12T10:09:27.544 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 2667, 2023-12-12T10:09:27.545 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 2668 2023-12-12T10:09:27.545 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.545 INFO:teuthology.orchestra.run.smithi173.stdout: "op": 2667, 2023-12-12T10:09:27.545 INFO:teuthology.orchestra.run.smithi173.stdout: "op_r": 2667, 2023-12-12T10:09:27.545 INFO:teuthology.orchestra.run.smithi173.stdout: "op_w": 0, 2023-12-12T10:09:27.545 INFO:teuthology.orchestra.run.smithi173.stdout: "op_rmw": 0, 2023-12-12T10:09:27.545 INFO:teuthology.orchestra.run.smithi173.stdout: "op_pg": 0, 2023-12-12T10:09:27.545 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_stat": 1722, 2023-12-12T10:09:27.545 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_create": 0, 2023-12-12T10:09:27.546 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_read": 943, 2023-12-12T10:09:27.546 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_write": 0, 2023-12-12T10:09:27.546 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writefull": 0, 2023-12-12T10:09:27.546 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_writesame": 0, 2023-12-12T10:09:27.546 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_append": 0, 2023-12-12T10:09:27.546 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_zero": 0, 2023-12-12T10:09:27.546 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_truncate": 0, 2023-12-12T10:09:27.546 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_delete": 0, 2023-12-12T10:09:27.547 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_mapext": 0, 2023-12-12T10:09:27.547 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_sparse_read": 0, 2023-12-12T10:09:27.547 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_clonerange": 0, 2023-12-12T10:09:27.547 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_getxattr": 1, 2023-12-12T10:09:27.547 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_setxattr": 0, 2023-12-12T10:09:27.547 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_cmpxattr": 0, 2023-12-12T10:09:27.547 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_rmxattr": 0, 2023-12-12T10:09:27.547 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_resetxattrs": 0, 2023-12-12T10:09:27.547 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_call": 0, 2023-12-12T10:09:27.548 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_watch": 0, 2023-12-12T10:09:27.548 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_notify": 0, 2023-12-12T10:09:27.548 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_src_cmpxattr": 0, 2023-12-12T10:09:27.548 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls": 0, 2023-12-12T10:09:27.548 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_pgls_filter": 0, 2023-12-12T10:09:27.548 INFO:teuthology.orchestra.run.smithi173.stdout: "osdop_other": 0, 2023-12-12T10:09:27.548 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_active": 0, 2023-12-12T10:09:27.548 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_send": 0, 2023-12-12T10:09:27.548 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_resend": 0, 2023-12-12T10:09:27.549 INFO:teuthology.orchestra.run.smithi173.stdout: "linger_ping": 0, 2023-12-12T10:09:27.549 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_active": 0, 2023-12-12T10:09:27.549 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_send": 0, 2023-12-12T10:09:27.549 INFO:teuthology.orchestra.run.smithi173.stdout: "poolop_resend": 0, 2023-12-12T10:09:27.549 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_active": 0, 2023-12-12T10:09:27.549 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_send": 0, 2023-12-12T10:09:27.549 INFO:teuthology.orchestra.run.smithi173.stdout: "poolstat_resend": 0, 2023-12-12T10:09:27.549 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_active": 0, 2023-12-12T10:09:27.549 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_send": 0, 2023-12-12T10:09:27.550 INFO:teuthology.orchestra.run.smithi173.stdout: "statfs_resend": 0, 2023-12-12T10:09:27.550 INFO:teuthology.orchestra.run.smithi173.stdout: "command_active": 0, 2023-12-12T10:09:27.550 INFO:teuthology.orchestra.run.smithi173.stdout: "command_send": 0, 2023-12-12T10:09:27.550 INFO:teuthology.orchestra.run.smithi173.stdout: "command_resend": 0, 2023-12-12T10:09:27.550 INFO:teuthology.orchestra.run.smithi173.stdout: "map_epoch": 99, 2023-12-12T10:09:27.550 INFO:teuthology.orchestra.run.smithi173.stdout: "map_full": 0, 2023-12-12T10:09:27.550 INFO:teuthology.orchestra.run.smithi173.stdout: "map_inc": 12, 2023-12-12T10:09:27.550 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_sessions": 11, 2023-12-12T10:09:27.550 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_open": 11, 2023-12-12T10:09:27.551 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_session_close": 0, 2023-12-12T10:09:27.551 INFO:teuthology.orchestra.run.smithi173.stdout: "osd_laggy": 0, 2023-12-12T10:09:27.551 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_wr": 0, 2023-12-12T10:09:27.551 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_rd": 2, 2023-12-12T10:09:27.551 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_del": 0 2023-12-12T10:09:27.551 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.551 INFO:teuthology.orchestra.run.smithi173.stdout: "oft": { 2023-12-12T10:09:27.551 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_objs": 0, 2023-12-12T10:09:27.551 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_kv_pairs": 0, 2023-12-12T10:09:27.552 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_updates": 0, 2023-12-12T10:09:27.552 INFO:teuthology.orchestra.run.smithi173.stdout: "omap_total_removes": 0 2023-12-12T10:09:27.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.552 INFO:teuthology.orchestra.run.smithi173.stdout: "osd-slow-ops": { 2023-12-12T10:09:27.552 INFO:teuthology.orchestra.run.smithi173.stdout: "slow_ops_count": 0 2023-12-12T10:09:27.552 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.552 INFO:teuthology.orchestra.run.smithi173.stdout: "purge_queue": { 2023-12-12T10:09:27.552 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops": 0, 2023-12-12T10:09:27.552 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_ops_high_water": 0, 2023-12-12T10:09:27.553 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing": 0, 2023-12-12T10:09:27.553 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executing_high_water": 0, 2023-12-12T10:09:27.553 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_executed": 0, 2023-12-12T10:09:27.553 INFO:teuthology.orchestra.run.smithi173.stdout: "pq_item_in_journal": 0 2023-12-12T10:09:27.553 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.553 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-12T10:09:27.553 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:27.553 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2023-12-12T10:09:27.553 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:27.554 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 2935, 2023-12-12T10:09:27.554 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 282772664, 2023-12-12T10:09:27.554 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:27.554 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 2935, 2023-12-12T10:09:27.554 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:27.554 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:27.554 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2935, 2023-12-12T10:09:27.554 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 282772664, 2023-12-12T10:09:27.554 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:27.555 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.555 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.555 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.555 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:27.555 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.555 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_bytes": { 2023-12-12T10:09:27.555 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:27.555 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 104857600, 2023-12-12T10:09:27.556 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:27.556 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:27.556 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:27.556 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:27.556 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:27.556 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 2667, 2023-12-12T10:09:27.556 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 280704711, 2023-12-12T10:09:27.556 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2667, 2023-12-12T10:09:27.556 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 280704711, 2023-12-12T10:09:27.557 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:27.557 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.557 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.557 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.557 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:27.557 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.557 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-objecter_ops": { 2023-12-12T10:09:27.557 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:27.557 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 1024, 2023-12-12T10:09:27.558 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:27.558 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:27.558 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:27.558 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:27.558 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:27.558 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 2667, 2023-12-12T10:09:27.558 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 2667, 2023-12-12T10:09:27.558 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 2667, 2023-12-12T10:09:27.558 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 2667, 2023-12-12T10:09:27.559 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:27.559 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.559 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.559 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.559 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:27.559 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.559 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle": { 2023-12-12T10:09:27.559 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:27.559 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2023-12-12T10:09:27.560 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:27.560 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:27.560 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:27.560 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:27.560 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:27.560 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:27.560 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:27.560 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2023-12-12T10:09:27.560 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2023-12-12T10:09:27.561 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:27.561 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.561 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.561 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.561 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:27.561 INFO:teuthology.orchestra.run.smithi173.stdout: }, 2023-12-12T10:09:27.561 INFO:teuthology.orchestra.run.smithi173.stdout: "throttle-write_buf_throttle-0x5601b4272960": { 2023-12-12T10:09:27.561 INFO:teuthology.orchestra.run.smithi173.stdout: "val": 0, 2023-12-12T10:09:27.561 INFO:teuthology.orchestra.run.smithi173.stdout: "max": 3758096384, 2023-12-12T10:09:27.562 INFO:teuthology.orchestra.run.smithi173.stdout: "get_started": 0, 2023-12-12T10:09:27.562 INFO:teuthology.orchestra.run.smithi173.stdout: "get": 0, 2023-12-12T10:09:27.562 INFO:teuthology.orchestra.run.smithi173.stdout: "get_sum": 0, 2023-12-12T10:09:27.562 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_fail": 0, 2023-12-12T10:09:27.562 INFO:teuthology.orchestra.run.smithi173.stdout: "get_or_fail_success": 0, 2023-12-12T10:09:27.562 INFO:teuthology.orchestra.run.smithi173.stdout: "take": 0, 2023-12-12T10:09:27.562 INFO:teuthology.orchestra.run.smithi173.stdout: "take_sum": 0, 2023-12-12T10:09:27.562 INFO:teuthology.orchestra.run.smithi173.stdout: "put": 0, 2023-12-12T10:09:27.563 INFO:teuthology.orchestra.run.smithi173.stdout: "put_sum": 0, 2023-12-12T10:09:27.563 INFO:teuthology.orchestra.run.smithi173.stdout: "wait": { 2023-12-12T10:09:27.563 INFO:teuthology.orchestra.run.smithi173.stdout: "avgcount": 0, 2023-12-12T10:09:27.563 INFO:teuthology.orchestra.run.smithi173.stdout: "sum": 0.000000000, 2023-12-12T10:09:27.563 INFO:teuthology.orchestra.run.smithi173.stdout: "avgtime": 0.000000000 2023-12-12T10:09:27.563 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:27.563 INFO:teuthology.orchestra.run.smithi173.stdout: } 2023-12-12T10:09:27.563 INFO:teuthology.orchestra.run.smithi173.stdout:} 2023-12-12T10:09:27.904 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2023-12-12T10:09:27.905 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2023-12-12T10:09:27.905 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2023-12-12T10:09:27.905 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2023-12-12T10:09:27.905 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2023-12-12T10:09:27.905 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2023-12-12T10:09:27.905 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2023-12-12T10:09:27.905 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2023-12-12T10:09:27.906 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-12T10:09:27.918 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-12T10:09:27.918 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T10:09:27.918 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T10:09:28.032 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2023-12-12T10:09:28.033 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-12T10:09:28.068 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T10:09:28.312 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.309+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/2772570786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d4105a90 msgr2=0x7fe4d4105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:28.336 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.309+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/2772570786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d4105a90 0x7fe4d4105e70 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400aaa0 tx=0x7fe4c400adb0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.311+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/2772570786 shutdown_connections 2023-12-12T10:09:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.311+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/2772570786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4d4100440 0x7fe4d410c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.311+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/2772570786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d4105a90 0x7fe4d4105e70 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.311+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/2772570786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d40ffa80 0x7fe4d40fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.311+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/2772570786 >> 172.21.15.42:0/2772570786 conn(0x7fe4d4076870 msgr2=0x7fe4d40fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.315+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/2772570786 shutdown_connections 2023-12-12T10:09:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.316+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/2772570786 wait complete. 2023-12-12T10:09:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.317+0000 7fe4d9bb3700 1 Processor -- start 2023-12-12T10:09:28.339 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.317+0000 7fe4d9bb3700 1 -- start start 2023-12-12T10:09:28.340 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d9bb3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4d4100440 0x7fe4d406d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:28.340 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d9bb3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d4105a90 0x7fe4d406c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:28.340 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d9bb3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d406c940 0x7fe4d406cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:28.340 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d9bb3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d41ae910 con 0x7fe4d406c940 2023-12-12T10:09:28.340 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d9bb3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d41aea80 con 0x7fe4d4105a90 2023-12-12T10:09:28.340 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d9bb3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d41aec40 con 0x7fe4d4100440 2023-12-12T10:09:28.340 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d4105a90 0x7fe4d406c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:28.340 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d406c940 0x7fe4d406cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:28.341 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d4105a90 0x7fe4d406c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:40142/0 (socket says 172.21.15.42:40142) 2023-12-12T10:09:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d3fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d406c940 0x7fe4d406cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37086/0 (socket says 172.21.15.42:37086) 2023-12-12T10:09:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d2ffd700 1 -- 172.21.15.42:0/3859566183 learned_addr learned my addr 172.21.15.42:0/3859566183 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d37fe700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4d4100440 0x7fe4d406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d2ffd700 1 -- 172.21.15.42:0/3859566183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4d4100440 msgr2=0x7fe4d406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d2ffd700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4d4100440 0x7fe4d406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d2ffd700 1 -- 172.21.15.42:0/3859566183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d406c940 msgr2=0x7fe4d406cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d2ffd700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d406c940 0x7fe4d406cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.342 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.318+0000 7fe4d2ffd700 1 -- 172.21.15.42:0/3859566183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c400a700 con 0x7fe4d4105a90 2023-12-12T10:09:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.319+0000 7fe4d2ffd700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d4105a90 0x7fe4d406c400 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00e870 tx=0x7fe4bc00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.319+0000 7fe4d0ff9700 1 -- 172.21.15.42:0/3859566183 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc009730 con 0x7fe4d4105a90 2023-12-12T10:09:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.319+0000 7fe4d0ff9700 1 -- 172.21.15.42:0/3859566183 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4bc010dc0 con 0x7fe4d4105a90 2023-12-12T10:09:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.319+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d406dfe0 con 0x7fe4d4105a90 2023-12-12T10:09:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.319+0000 7fe4d0ff9700 1 -- 172.21.15.42:0/3859566183 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc00f420 con 0x7fe4d4105a90 2023-12-12T10:09:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.319+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d406e4a0 con 0x7fe4d4105a90 2023-12-12T10:09:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.320+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d4108080 con 0x7fe4d4105a90 2023-12-12T10:09:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.321+0000 7fe4d0ff9700 1 -- 172.21.15.42:0/3859566183 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fe4bc018690 con 0x7fe4d4105a90 2023-12-12T10:09:28.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.322+0000 7fe4d0ff9700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe4c00a15f0 0x7fe4c00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:28.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.322+0000 7fe4d0ff9700 1 -- 172.21.15.42:0/3859566183 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe4bc014070 con 0x7fe4d4105a90 2023-12-12T10:09:28.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.322+0000 7fe4d37fe700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe4c00a15f0 0x7fe4c00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:28.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.323+0000 7fe4d37fe700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe4c00a15f0 0x7fe4c00a3ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400c6d0 tx=0x7fe4c4023040 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:28.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.329+0000 7fe4d0ff9700 1 -- 172.21.15.42:0/3859566183 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe4bc0522f0 con 0x7fe4d4105a90 2023-12-12T10:09:28.517 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:28 smithi042 ceph-mon[98894]: pgmap v335: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-12-12T10:09:28.629 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.626+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4d410f170 con 0x7fe4d4105a90 2023-12-12T10:09:28.631 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.628+0000 7fe4d0ff9700 1 -- 172.21.15.42:0/3859566183 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+25310 (secure 0 0 0) 0x7fe4bc099f70 con 0x7fe4d4105a90 2023-12-12T10:09:28.632 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T10:09:28.632 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":99,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T10:09:07.830289+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2637927159":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/2112736882":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/3552425799":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/3987075096":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/2637927159":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3046151373":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T10:09:28.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.632+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe4c00a15f0 msgr2=0x7fe4c00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:28.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.632+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe4c00a15f0 0x7fe4c00a3ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe4c400c6d0 tx=0x7fe4c4023040 comp rx=0 tx=0).stop 2023-12-12T10:09:28.634 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.632+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d4105a90 msgr2=0x7fe4d406c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:28.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.632+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d4105a90 0x7fe4d406c400 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00e870 tx=0x7fe4bc00eb80 comp rx=0 tx=0).stop 2023-12-12T10:09:28.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.633+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 shutdown_connections 2023-12-12T10:09:28.635 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.633+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4d406c940 0x7fe4d406cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.633+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4d4100440 0x7fe4d406d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.633+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe4c00a15f0 0x7fe4c00a3ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.633+0000 7fe4d9bb3700 1 --2- 172.21.15.42:0/3859566183 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4d4105a90 0x7fe4d406c400 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.633+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 >> 172.21.15.42:0/3859566183 conn(0x7fe4d4076870 msgr2=0x7fe4d4102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:28.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.633+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 shutdown_connections 2023-12-12T10:09:28.636 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.633+0000 7fe4d9bb3700 1 -- 172.21.15.42:0/3859566183 wait complete. 2023-12-12T10:09:28.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-12T09:52:42.268062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-12T09:56:35.884237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-12T09:56:37.283813+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-12T09:56:40.918865+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 16, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-12T10:09:28.654 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-12T10:09:28.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:28 smithi173 ceph-mon[103308]: pgmap v335: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-12-12T10:09:28.857 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.854+0000 7fadcba5e700 1 -- 172.21.15.42:0/116194426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc410a890 msgr2=0x7fadc4112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:28.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.854+0000 7fadcba5e700 1 --2- 172.21.15.42:0/116194426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc410a890 0x7fadc4112fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009990 tx=0x7fadc0009ca0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.854+0000 7fadcba5e700 1 -- 172.21.15.42:0/116194426 shutdown_connections 2023-12-12T10:09:28.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.854+0000 7fadcba5e700 1 --2- 172.21.15.42:0/116194426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc4113520 0x7fadc4115910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.854+0000 7fadcba5e700 1 --2- 172.21.15.42:0/116194426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc4109f70 0x7fadc410a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.854+0000 7fadcba5e700 1 --2- 172.21.15.42:0/116194426 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc410a890 0x7fadc4112fe0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.854+0000 7fadcba5e700 1 -- 172.21.15.42:0/116194426 >> 172.21.15.42:0/116194426 conn(0x7fadc406c610 msgr2=0x7fadc406ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:28.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.854+0000 7fadcba5e700 1 -- 172.21.15.42:0/116194426 shutdown_connections 2023-12-12T10:09:28.858 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.854+0000 7fadcba5e700 1 -- 172.21.15.42:0/116194426 wait complete. 2023-12-12T10:09:28.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadcba5e700 1 Processor -- start 2023-12-12T10:09:28.859 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadcba5e700 1 -- start start 2023-12-12T10:09:28.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadcba5e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc4109f70 0x7fadc40700b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:28.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadcba5e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc410a890 0x7fadc40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:28.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadcba5e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc4113520 0x7fadc4070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:28.860 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadcba5e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41ae900 con 0x7fadc4109f70 2023-12-12T10:09:28.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadcba5e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41aea70 con 0x7fadc410a890 2023-12-12T10:09:28.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadcba5e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc41aec30 con 0x7fadc4113520 2023-12-12T10:09:28.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadc97fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc4109f70 0x7fadc40700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:28.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadc9ffb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc4113520 0x7fadc4070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:28.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadc97fa700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc4109f70 0x7fadc40700b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37094/0 (socket says 172.21.15.42:37094) 2023-12-12T10:09:28.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.855+0000 7fadc97fa700 1 -- 172.21.15.42:0/4116662461 learned_addr learned my addr 172.21.15.42:0/4116662461 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:28.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.856+0000 7fadc8ff9700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc410a890 0x7fadc40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:28.861 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.856+0000 7fadc97fa700 1 -- 172.21.15.42:0/4116662461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc4113520 msgr2=0x7fadc4070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:28.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.856+0000 7fadc97fa700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc4113520 0x7fadc4070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.856+0000 7fadc97fa700 1 -- 172.21.15.42:0/4116662461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc410a890 msgr2=0x7fadc40705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:28.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.856+0000 7fadc97fa700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc410a890 0x7fadc40705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:28.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.856+0000 7fadc97fa700 1 -- 172.21.15.42:0/4116662461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00095f0 con 0x7fadc4109f70 2023-12-12T10:09:28.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.856+0000 7fadc8ff9700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc410a890 0x7fadc40705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-12T10:09:28.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.856+0000 7fadc97fa700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc4109f70 0x7fadc40700b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00ba50 tx=0x7fadbc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:28.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.858+0000 7fadba7fc700 1 -- 172.21.15.42:0/4116662461 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00c930 con 0x7fadc4109f70 2023-12-12T10:09:28.862 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.858+0000 7fadba7fc700 1 -- 172.21.15.42:0/4116662461 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadbc014440 con 0x7fadc4109f70 2023-12-12T10:09:28.863 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.858+0000 7fadba7fc700 1 -- 172.21.15.42:0/4116662461 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0075d0 con 0x7fadc4109f70 2023-12-12T10:09:28.863 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.859+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc41aef10 con 0x7fadc4109f70 2023-12-12T10:09:28.863 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.859+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc41af340 con 0x7fadc4109f70 2023-12-12T10:09:28.863 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.860+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc410f740 con 0x7fadc4109f70 2023-12-12T10:09:28.865 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.862+0000 7fadba7fc700 1 -- 172.21.15.42:0/4116662461 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fadbc0145b0 con 0x7fadc4109f70 2023-12-12T10:09:28.865 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.863+0000 7fadba7fc700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fadb00a1740 0x7fadb00a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:28.866 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.863+0000 7fadba7fc700 1 -- 172.21.15.42:0/4116662461 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fadbc006e40 con 0x7fadc4109f70 2023-12-12T10:09:28.866 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.863+0000 7fadc8ff9700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fadb00a1740 0x7fadb00a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:28.866 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.864+0000 7fadc8ff9700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fadb00a1740 0x7fadb00a3c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fadc40717e0 tx=0x7fadc000b540 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:28.870 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:28.867+0000 7fadba7fc700 1 -- 172.21.15.42:0/4116662461 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fadbc0962e0 con 0x7fadc4109f70 2023-12-12T10:09:28.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:28 smithi111 ceph-mon[102565]: pgmap v335: 129 pgs: 16 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 63 active+clean; 717 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-12-12T10:09:29.162 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.160+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fadc4108c40 con 0x7fadc4109f70 2023-12-12T10:09:29.163 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.160+0000 7fadba7fc700 1 -- 172.21.15.42:0/4116662461 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fadbc0190a0 con 0x7fadc4109f70 2023-12-12T10:09:29.163 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2023-12-12T10:09:29.166 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fadb00a1740 msgr2=0x7fadb00a3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:29.166 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fadb00a1740 0x7fadb00a3c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fadc40717e0 tx=0x7fadc000b540 comp rx=0 tx=0).stop 2023-12-12T10:09:29.166 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc4109f70 msgr2=0x7fadc40700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:29.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc4109f70 0x7fadc40700b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fadbc00ba50 tx=0x7fadbc00bd60 comp rx=0 tx=0).stop 2023-12-12T10:09:29.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 shutdown_connections 2023-12-12T10:09:29.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadc4113520 0x7fadc4070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadc4109f70 0x7fadc40700b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fadb00a1740 0x7fadb00a3c00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 --2- 172.21.15.42:0/4116662461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fadc410a890 0x7fadc40705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 >> 172.21.15.42:0/4116662461 conn(0x7fadc406c610 msgr2=0x7fadc4109240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:29.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 shutdown_connections 2023-12-12T10:09:29.167 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.163+0000 7fadcba5e700 1 -- 172.21.15.42:0/4116662461 wait complete. 2023-12-12T10:09:29.180 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-12T10:09:29.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.335+0000 7f866da52700 1 -- 172.21.15.42:0/3357294838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8668101ee0 msgr2=0x7f866810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:29.343 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.335+0000 7f866da52700 1 --2- 172.21.15.42:0/3357294838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8668101ee0 0x7f866810e3a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8660009f90 tx=0x7f866000b330 comp rx=0 tx=0).stop 2023-12-12T10:09:29.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.335+0000 7f866da52700 1 -- 172.21.15.42:0/3357294838 shutdown_connections 2023-12-12T10:09:29.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.335+0000 7f866da52700 1 --2- 172.21.15.42:0/3357294838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86681075e0 0x7f86681079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.335+0000 7f866da52700 1 --2- 172.21.15.42:0/3357294838 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8668101ee0 0x7f866810e3a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.335+0000 7f866da52700 1 --2- 172.21.15.42:0/3357294838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8668101520 0x7f86681019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.344 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.335+0000 7f866da52700 1 -- 172.21.15.42:0/3357294838 >> 172.21.15.42:0/3357294838 conn(0x7f86680fd010 msgr2=0x7f86680ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:29.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.335+0000 7f866da52700 1 -- 172.21.15.42:0/3357294838 shutdown_connections 2023-12-12T10:09:29.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.335+0000 7f866da52700 1 -- 172.21.15.42:0/3357294838 wait complete. 2023-12-12T10:09:29.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f866da52700 1 Processor -- start 2023-12-12T10:09:29.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f866da52700 1 -- start start 2023-12-12T10:09:29.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f866da52700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8668101520 0x7f8668197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:29.345 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f866da52700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8668101ee0 0x7f8668197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:29.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f866da52700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86681075e0 0x7f866819c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:29.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f866da52700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866819c950 con 0x7f86681075e0 2023-12-12T10:09:29.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f866da52700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866819cac0 con 0x7f8668101520 2023-12-12T10:09:29.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f866da52700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866819cc80 con 0x7f8668101ee0 2023-12-12T10:09:29.346 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f8666ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8668101520 0x7f8668197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:29.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f8666ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8668101520 0x7f8668197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:40168/0 (socket says 172.21.15.42:40168) 2023-12-12T10:09:29.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f8666ffd700 1 -- 172.21.15.42:0/2488840337 learned_addr learned my addr 172.21.15.42:0/2488840337 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:29.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f865ffff700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8668101ee0 0x7f8668197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:29.347 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f86677fe700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86681075e0 0x7f866819c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:29.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f8666ffd700 1 -- 172.21.15.42:0/2488840337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8668101ee0 msgr2=0x7f8668197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:29.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f8666ffd700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8668101ee0 0x7f8668197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f8666ffd700 1 -- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86681075e0 msgr2=0x7f866819c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:29.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f8666ffd700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86681075e0 0x7f866819c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.348 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.336+0000 7f8666ffd700 1 -- 172.21.15.42:0/2488840337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86500095f0 con 0x7f8668101520 2023-12-12T10:09:29.349 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.337+0000 7f8666ffd700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8668101520 0x7f8668197a70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f865800e9a0 tx=0x7f865800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:29.349 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.337+0000 7f866ca50700 1 -- 172.21.15.42:0/2488840337 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658009790 con 0x7f8668101520 2023-12-12T10:09:29.349 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.337+0000 7f866ca50700 1 -- 172.21.15.42:0/2488840337 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8658010430 con 0x7f8668101520 2023-12-12T10:09:29.349 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.337+0000 7f866ca50700 1 -- 172.21.15.42:0/2488840337 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8658018680 con 0x7f8668101520 2023-12-12T10:09:29.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.337+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8660009c40 con 0x7f8668101520 2023-12-12T10:09:29.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.337+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86681a4630 con 0x7f8668101520 2023-12-12T10:09:29.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.338+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8668109a80 con 0x7f8668101520 2023-12-12T10:09:29.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.339+0000 7f866ca50700 1 -- 172.21.15.42:0/2488840337 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f86580105a0 con 0x7f8668101520 2023-12-12T10:09:29.350 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.340+0000 7f866ca50700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f86540a1670 0x7f86540a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:29.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.340+0000 7f866ca50700 1 -- 172.21.15.42:0/2488840337 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f8658014070 con 0x7f8668101520 2023-12-12T10:09:29.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.340+0000 7f865ffff700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f86540a1670 0x7f86540a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:29.351 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.340+0000 7f865ffff700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f86540a1670 0x7f86540a3b30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8668198fc0 tx=0x7f8650009310 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:29.352 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.345+0000 7f866ca50700 1 -- 172.21.15.42:0/2488840337 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f8658096580 con 0x7f8668101520 2023-12-12T10:09:29.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.610+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8668067a80 con 0x7f8668101520 2023-12-12T10:09:29.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.611+0000 7f866ca50700 1 -- 172.21.15.42:0/2488840337 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8658099ba0 con 0x7f8668101520 2023-12-12T10:09:29.614 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T10:09:29.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.615+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f86540a1670 msgr2=0x7f86540a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:29.618 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.615+0000 7f866da52700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f86540a1670 0x7f86540a3b30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8668198fc0 tx=0x7f8650009310 comp rx=0 tx=0).stop 2023-12-12T10:09:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.615+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8668101520 msgr2=0x7f8668197a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.615+0000 7f866da52700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8668101520 0x7f8668197a70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f865800e9a0 tx=0x7f865800ecb0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.615+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 shutdown_connections 2023-12-12T10:09:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.615+0000 7f866da52700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f86540a1670 0x7f86540a3b30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.615+0000 7f866da52700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8668101520 0x7f8668197a70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.616+0000 7f866da52700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f86681075e0 0x7f866819c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.616+0000 7f866da52700 1 --2- 172.21.15.42:0/2488840337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8668101ee0 0x7f8668197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.619 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.616+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 >> 172.21.15.42:0/2488840337 conn(0x7f86680fd010 msgr2=0x7f86680fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.616+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 shutdown_connections 2023-12-12T10:09:29.620 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.616+0000 7f866da52700 1 -- 172.21.15.42:0/2488840337 wait complete. 2023-12-12T10:09:29.633 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-12T10:09:29.792 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 -- 172.21.15.42:0/691189981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4524101ee0 msgr2=0x7f452410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:29.793 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 --2- 172.21.15.42:0/691189981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4524101ee0 0x7f452410e3a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f451c00b0d0 tx=0x7f451c00b3e0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.793 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 -- 172.21.15.42:0/691189981 shutdown_connections 2023-12-12T10:09:29.793 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 --2- 172.21.15.42:0/691189981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45241075e0 0x7f45241079c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.794 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 --2- 172.21.15.42:0/691189981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4524101ee0 0x7f452410e3a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.794 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 --2- 172.21.15.42:0/691189981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4524101520 0x7f45241019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.794 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 -- 172.21.15.42:0/691189981 >> 172.21.15.42:0/691189981 conn(0x7f45240fd010 msgr2=0x7f45240ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:29.794 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 -- 172.21.15.42:0/691189981 shutdown_connections 2023-12-12T10:09:29.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 -- 172.21.15.42:0/691189981 wait complete. 2023-12-12T10:09:29.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.786+0000 7f4529b09700 1 Processor -- start 2023-12-12T10:09:29.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f4529b09700 1 -- start start 2023-12-12T10:09:29.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f4529b09700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4524101520 0x7f452419f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:29.795 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f4529b09700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4524101ee0 0x7f452419fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:29.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f4529b09700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45241075e0 0x7f4524199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:29.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f4529b09700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452419a140 con 0x7f4524101ee0 2023-12-12T10:09:29.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f4529b09700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452419a2b0 con 0x7f45241075e0 2023-12-12T10:09:29.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f4529b09700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452419a470 con 0x7f4524101520 2023-12-12T10:09:29.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f45237fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4524101520 0x7f452419f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:29.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f4523fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45241075e0 0x7f4524199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:29.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f45237fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4524101520 0x7f452419f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.42:52106/0 (socket says 172.21.15.42:52106) 2023-12-12T10:09:29.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f45237fe700 1 -- 172.21.15.42:0/3823107700 learned_addr learned my addr 172.21.15.42:0/3823107700 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:29.796 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.787+0000 7f4523fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45241075e0 0x7f4524199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.42:40184/0 (socket says 172.21.15.42:40184) 2023-12-12T10:09:29.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f4523fff700 1 -- 172.21.15.42:0/3823107700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4524101520 msgr2=0x7f452419f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:29.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f4523fff700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4524101520 0x7f452419f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f4523fff700 1 -- 172.21.15.42:0/3823107700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4524101ee0 msgr2=0x7f452419fef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-12T10:09:29.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f4523fff700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4524101ee0 0x7f452419fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:29.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f4523fff700 1 -- 172.21.15.42:0/3823107700 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4514009520 con 0x7f45241075e0 2023-12-12T10:09:29.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f4523fff700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45241075e0 0x7f4524199ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f451c00b770 tx=0x7f451c00bcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:29.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f45217fa700 1 -- 172.21.15.42:0/3823107700 <== mon.2 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c00e040 con 0x7f45241075e0 2023-12-12T10:09:29.797 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f45217fa700 1 -- 172.21.15.42:0/3823107700 <== mon.2 v2:172.21.15.111:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f451c005030 con 0x7f45241075e0 2023-12-12T10:09:29.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451c009d00 con 0x7f45241075e0 2023-12-12T10:09:29.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f45217fa700 1 -- 172.21.15.42:0/3823107700 <== mon.2 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c01b6a0 con 0x7f45241075e0 2023-12-12T10:09:29.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.788+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45241a67c0 con 0x7f45241075e0 2023-12-12T10:09:29.798 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.789+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4524102c40 con 0x7f45241075e0 2023-12-12T10:09:29.799 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.790+0000 7f45217fa700 1 -- 172.21.15.42:0/3823107700 <== mon.2 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f451c0051a0 con 0x7f45241075e0 2023-12-12T10:09:29.799 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.791+0000 7f45217fa700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f45100a1670 0x7f45100a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:29.799 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.791+0000 7f45237fe700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f45100a1670 0x7f45100a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:29.799 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.791+0000 7f45217fa700 1 -- 172.21.15.42:0/3823107700 <== mon.2 v2:172.21.15.111:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f451c0ce320 con 0x7f45241075e0 2023-12-12T10:09:29.799 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.792+0000 7f45237fe700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f45100a1670 0x7f45100a3b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4514009ef0 tx=0x7f4514009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:29.800 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:29.796+0000 7f45217fa700 1 -- 172.21.15.42:0/3823107700 <== mon.2 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f451c0972d0 con 0x7f45241075e0 2023-12-12T10:09:29.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:29 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3859566183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T10:09:29.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:29 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/4116662461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T10:09:29.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:29 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3859566183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T10:09:29.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:29 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/4116662461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T10:09:29.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:29 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3859566183' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T10:09:29.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:29 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/4116662461' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-12T10:09:30.055 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.052+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f452419afa0 con 0x7f45241075e0 2023-12-12T10:09:30.055 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.053+0000 7f45217fa700 1 -- 172.21.15.42:0/3823107700 <== mon.2 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f451c00a040 con 0x7f45241075e0 2023-12-12T10:09:30.056 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2023-12-12T10:09:30.058 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f45100a1670 msgr2=0x7f45100a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.058 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f45100a1670 0x7f45100a3b30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4514009ef0 tx=0x7f4514009260 comp rx=0 tx=0).stop 2023-12-12T10:09:30.058 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45241075e0 msgr2=0x7f4524199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.058 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45241075e0 0x7f4524199ae0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f451c00b770 tx=0x7f451c00bcd0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.058 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 shutdown_connections 2023-12-12T10:09:30.058 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f45100a1670 0x7f45100a3b30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.058 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45241075e0 0x7f4524199ae0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.059 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4524101520 0x7f452419f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.059 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 --2- 172.21.15.42:0/3823107700 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4524101ee0 0x7f452419fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.059 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 >> 172.21.15.42:0/3823107700 conn(0x7f45240fd010 msgr2=0x7f45240fe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:30.059 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 shutdown_connections 2023-12-12T10:09:30.059 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.055+0000 7f4529b09700 1 -- 172.21.15.42:0/3823107700 wait complete. 2023-12-12T10:09:30.073 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-12T10:09:30.229 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.225+0000 7fce53c0e700 1 -- 172.21.15.42:0/3684851615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c1064c0 msgr2=0x7fce4c1068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.225+0000 7fce53c0e700 1 --2- 172.21.15.42:0/3684851615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c1064c0 0x7fce4c1068a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fce48009990 tx=0x7fce48009ca0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.225+0000 7fce53c0e700 1 -- 172.21.15.42:0/3684851615 shutdown_connections 2023-12-12T10:09:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.225+0000 7fce53c0e700 1 --2- 172.21.15.42:0/3684851615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce4c100de0 0x7fce4c10d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.225+0000 7fce53c0e700 1 --2- 172.21.15.42:0/3684851615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c1064c0 0x7fce4c1068a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.225+0000 7fce53c0e700 1 --2- 172.21.15.42:0/3684851615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce4c100420 0x7fce4c1008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.225+0000 7fce53c0e700 1 -- 172.21.15.42:0/3684851615 >> 172.21.15.42:0/3684851615 conn(0x7fce4c0775a0 msgr2=0x7fce4c0779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:30.230 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.225+0000 7fce53c0e700 1 -- 172.21.15.42:0/3684851615 shutdown_connections 2023-12-12T10:09:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.225+0000 7fce53c0e700 1 -- 172.21.15.42:0/3684851615 wait complete. 2023-12-12T10:09:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce53c0e700 1 Processor -- start 2023-12-12T10:09:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce53c0e700 1 -- start start 2023-12-12T10:09:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce53c0e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce4c100420 0x7fce4c195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce53c0e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c100de0 0x7fce4c195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce53c0e700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce4c1064c0 0x7fce4c198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce53c0e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c199590 con 0x7fce4c100de0 2023-12-12T10:09:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce53c0e700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c199700 con 0x7fce4c1064c0 2023-12-12T10:09:30.231 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce53c0e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4c1998c0 con 0x7fce4c100420 2023-12-12T10:09:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce511a9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c100de0 0x7fce4c195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.226+0000 7fce519aa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce4c100420 0x7fce4c195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce511a9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c100de0 0x7fce4c195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37138/0 (socket says 172.21.15.42:37138) 2023-12-12T10:09:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce511a9700 1 -- 172.21.15.42:0/1129893304 learned_addr learned my addr 172.21.15.42:0/1129893304 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce521ab700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce4c1064c0 0x7fce4c198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce519aa700 1 -- 172.21.15.42:0/1129893304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce4c1064c0 msgr2=0x7fce4c198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce519aa700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce4c1064c0 0x7fce4c198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce519aa700 1 -- 172.21.15.42:0/1129893304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c100de0 msgr2=0x7fce4c195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce519aa700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c100de0 0x7fce4c195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce519aa700 1 -- 172.21.15.42:0/1129893304 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480095f0 con 0x7fce4c100420 2023-12-12T10:09:30.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce519aa700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce4c100420 0x7fce4c195760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fce48000c00 tx=0x7fce48009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce42ffd700 1 -- 172.21.15.42:0/1129893304 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4801d070 con 0x7fce4c100420 2023-12-12T10:09:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4c199b40 con 0x7fce4c100420 2023-12-12T10:09:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce42ffd700 1 -- 172.21.15.42:0/1129893304 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce48021410 con 0x7fce4c100420 2023-12-12T10:09:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4c1a2500 con 0x7fce4c100420 2023-12-12T10:09:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.227+0000 7fce42ffd700 1 -- 172.21.15.42:0/1129893304 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce48017650 con 0x7fce4c100420 2023-12-12T10:09:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.228+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce300052f0 con 0x7fce4c100420 2023-12-12T10:09:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.230+0000 7fce42ffd700 1 -- 172.21.15.42:0/1129893304 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fce4800fb10 con 0x7fce4c100420 2023-12-12T10:09:30.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.231+0000 7fce42ffd700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce380a3880 0x7fce380a5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:30.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.231+0000 7fce42ffd700 1 -- 172.21.15.42:0/1129893304 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fce480cecd0 con 0x7fce4c100420 2023-12-12T10:09:30.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.231+0000 7fce511a9700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce380a3880 0x7fce380a5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:30.235 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.232+0000 7fce511a9700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce380a3880 0x7fce380a5d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce4c196cb0 tx=0x7fce3c00b500 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:30.236 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.234+0000 7fce42ffd700 1 -- 172.21.15.42:0/1129893304 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fce48097c80 con 0x7fce4c100420 2023-12-12T10:09:30.496 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.493+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fce30005b20 con 0x7fce4c100420 2023-12-12T10:09:30.496 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.493+0000 7fce42ffd700 1 -- 172.21.15.42:0/1129893304 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v99) v1 ==== 103+0+11 (secure 0 0 0) 0x7fce4809b2a0 con 0x7fce4c100420 2023-12-12T10:09:30.497 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2023-12-12T10:09:30.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce380a3880 msgr2=0x7fce380a5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.499 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce380a3880 0x7fce380a5d40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce4c196cb0 tx=0x7fce3c00b500 comp rx=0 tx=0).stop 2023-12-12T10:09:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce4c100420 msgr2=0x7fce4c195760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce4c100420 0x7fce4c195760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fce48000c00 tx=0x7fce48009580 comp rx=0 tx=0).stop 2023-12-12T10:09:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 shutdown_connections 2023-12-12T10:09:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fce380a3880 0x7fce380a5d40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fce4c1064c0 0x7fce4c198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fce4c100420 0x7fce4c195760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 --2- 172.21.15.42:0/1129893304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce4c100de0 0x7fce4c195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.500 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.496+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 >> 172.21.15.42:0/1129893304 conn(0x7fce4c0775a0 msgr2=0x7fce4c0fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.497+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 shutdown_connections 2023-12-12T10:09:30.501 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.497+0000 7fce53c0e700 1 -- 172.21.15.42:0/1129893304 wait complete. 2023-12-12T10:09:30.514 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-12T10:09:30.654 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.649+0000 7fa8da0f0700 1 -- 172.21.15.42:0/1740949569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4107670 msgr2=0x7fa8d4107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.649+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/1740949569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4107670 0x7fa8d4107a50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4009990 tx=0x7fa8c4009ca0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.649+0000 7fa8da0f0700 1 -- 172.21.15.42:0/1740949569 shutdown_connections 2023-12-12T10:09:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.649+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/1740949569 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d4068ff0 0x7fa8d410e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.649+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/1740949569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4107670 0x7fa8d4107a50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.649+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/1740949569 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8d4068650 0x7fa8d4068ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.649+0000 7fa8da0f0700 1 -- 172.21.15.42:0/1740949569 >> 172.21.15.42:0/1740949569 conn(0x7fa8d4077560 msgr2=0x7fa8d4077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.650+0000 7fa8da0f0700 1 -- 172.21.15.42:0/1740949569 shutdown_connections 2023-12-12T10:09:30.655 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.650+0000 7fa8da0f0700 1 -- 172.21.15.42:0/1740949569 wait complete. 2023-12-12T10:09:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.650+0000 7fa8da0f0700 1 Processor -- start 2023-12-12T10:09:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.650+0000 7fa8da0f0700 1 -- start start 2023-12-12T10:09:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8da0f0700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8d4068650 0x7fa8d419d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8da0f0700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4068ff0 0x7fa8d419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8da0f0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d4107670 0x7fa8d4197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8da0f0700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d4197ea0 con 0x7fa8d4068ff0 2023-12-12T10:09:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8da0f0700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d4198010 con 0x7fa8d4068650 2023-12-12T10:09:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8da0f0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d41981d0 con 0x7fa8d4107670 2023-12-12T10:09:30.656 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4068ff0 0x7fa8d419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d37fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8d4068650 0x7fa8d419d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d2ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4068ff0 0x7fa8d419dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37152/0 (socket says 172.21.15.42:37152) 2023-12-12T10:09:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d2ffd700 1 -- 172.21.15.42:0/3073209566 learned_addr learned my addr 172.21.15.42:0/3073209566 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d3fff700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d4107670 0x7fa8d4197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d2ffd700 1 -- 172.21.15.42:0/3073209566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d4107670 msgr2=0x7fa8d4197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d2ffd700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d4107670 0x7fa8d4197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d2ffd700 1 -- 172.21.15.42:0/3073209566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8d4068650 msgr2=0x7fa8d419d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.657 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d2ffd700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8d4068650 0x7fa8d419d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d2ffd700 1 -- 172.21.15.42:0/3073209566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40095f0 con 0x7fa8d4068ff0 2023-12-12T10:09:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d3fff700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d4107670 0x7fa8d4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:09:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d2ffd700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4068ff0 0x7fa8d419dc40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800d980 tx=0x7fa8c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d0ff9700 1 -- 172.21.15.42:0/3073209566 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8004220 con 0x7fa8d4068ff0 2023-12-12T10:09:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d41984b0 con 0x7fa8d4068ff0 2023-12-12T10:09:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d0ff9700 1 -- 172.21.15.42:0/3073209566 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8c8007640 con 0x7fa8d4068ff0 2023-12-12T10:09:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8d0ff9700 1 -- 172.21.15.42:0/3073209566 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c8005610 con 0x7fa8d4068ff0 2023-12-12T10:09:30.658 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.651+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d41a46f0 con 0x7fa8d4068ff0 2023-12-12T10:09:30.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.652+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d4109a20 con 0x7fa8d4068ff0 2023-12-12T10:09:30.659 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.654+0000 7fa8d0ff9700 1 -- 172.21.15.42:0/3073209566 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fa8c8004750 con 0x7fa8d4068ff0 2023-12-12T10:09:30.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.655+0000 7fa8d0ff9700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa8bc0a15f0 0x7fa8bc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:30.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.655+0000 7fa8d37fe700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa8bc0a15f0 0x7fa8bc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:30.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.655+0000 7fa8d0ff9700 1 -- 172.21.15.42:0/3073209566 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa8c80cd840 con 0x7fa8d4068ff0 2023-12-12T10:09:30.660 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.655+0000 7fa8d37fe700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa8bc0a15f0 0x7fa8bc0a3ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b5c0 tx=0x7fa8c4005960 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:30.661 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.658+0000 7fa8d0ff9700 1 -- 172.21.15.42:0/3073209566 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa8c8096760 con 0x7fa8d4068ff0 2023-12-12T10:09:30.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:30 smithi173 ceph-mon[103308]: pgmap v336: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 717 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:09:30.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:30 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2488840337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T10:09:30.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:30 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3823107700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T10:09:30.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:30 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/1129893304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T10:09:30.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.877+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa8d4068340 con 0x7fa8d4068ff0 2023-12-12T10:09:30.880 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.877+0000 7fa8d0ff9700 1 -- 172.21.15.42:0/3073209566 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v99) v1 ==== 81+0+945 (secure 0 0 0) 0x7fa8c8016080 con 0x7fa8d4068ff0 2023-12-12T10:09:30.880 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/2637927159 2023-12-13T09:58:36.825079+0000 2023-12-12T10:09:30.881 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2112736882 2023-12-13T09:58:36.825079+0000 2023-12-12T10:09:30.881 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3552425799 2023-12-13T09:58:36.825079+0000 2023-12-12T10:09:30.881 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3987075096 2023-12-13T09:58:36.825079+0000 2023-12-12T10:09:30.881 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/467026676 2023-12-13T09:50:27.727252+0000 2023-12-12T10:09:30.881 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2920147377 2023-12-13T09:50:27.727252+0000 2023-12-12T10:09:30.881 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/1525180467 2023-12-13T09:50:27.727252+0000 2023-12-12T10:09:30.881 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/1433305616 2023-12-13T09:49:58.206187+0000 2023-12-12T10:09:30.882 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/1433305616 2023-12-13T09:49:58.206187+0000 2023-12-12T10:09:30.882 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3856214912 2023-12-13T09:49:58.206187+0000 2023-12-12T10:09:30.882 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/1525180467 2023-12-13T09:50:27.727252+0000 2023-12-12T10:09:30.882 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1613862950 2023-12-13T09:50:27.727252+0000 2023-12-12T10:09:30.882 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/2637927159 2023-12-13T09:58:36.825079+0000 2023-12-12T10:09:30.882 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4273364865 2023-12-13T09:49:58.206187+0000 2023-12-12T10:09:30.882 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3046151373 2023-12-13T09:58:36.825079+0000 2023-12-12T10:09:30.882 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/2212533654 2023-12-13T09:49:58.206187+0000 2023-12-12T10:09:30.883 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.879+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa8bc0a15f0 msgr2=0x7fa8bc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.879+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa8bc0a15f0 0x7fa8bc0a3ab0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400b5c0 tx=0x7fa8c4005960 comp rx=0 tx=0).stop 2023-12-12T10:09:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.879+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4068ff0 msgr2=0x7fa8d419dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.879+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4068ff0 0x7fa8d419dc40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800d980 tx=0x7fa8c800dc90 comp rx=0 tx=0).stop 2023-12-12T10:09:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.880+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 shutdown_connections 2023-12-12T10:09:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.880+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa8bc0a15f0 0x7fa8bc0a3ab0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.880+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa8d4068650 0x7fa8d419d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.880+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d4107670 0x7fa8d4197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.880+0000 7fa8da0f0700 1 --2- 172.21.15.42:0/3073209566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8d4068ff0 0x7fa8d419dc40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:30.884 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.880+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 >> 172.21.15.42:0/3073209566 conn(0x7fa8d4077560 msgr2=0x7fa8d40ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:30.885 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.880+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 shutdown_connections 2023-12-12T10:09:30.885 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:30.880+0000 7fa8da0f0700 1 -- 172.21.15.42:0/3073209566 wait complete. 2023-12-12T10:09:30.885 INFO:teuthology.orchestra.run.smithi042.stderr:listed 16 entries 2023-12-12T10:09:30.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:30 smithi042 ceph-mon[98894]: pgmap v336: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 717 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:09:30.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:30 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2488840337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T10:09:30.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:30 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3823107700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T10:09:30.894 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:30 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/1129893304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T10:09:30.895 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-12T10:09:30.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T10:09:30.896 DEBUG:teuthology.orchestra.run.smithi042:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-12T10:09:30.996 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T10:09:30.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T10:09:30.997 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-12T10:09:30.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:30 smithi111 ceph-mon[102565]: pgmap v336: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 717 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 13 op/s 2023-12-12T10:09:30.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:30 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2488840337' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-12T10:09:31.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:30 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3823107700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-12T10:09:31.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:30 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/1129893304' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-12T10:09:31.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T10:09:31.154 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2023-12-12T10:09:31.360 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-12T10:09:31.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T10:09:31.360 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-12T10:09:31.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-12T10:09:31.513 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2023-12-12T10:09:31.627 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-12T10:09:31.639 INFO:tasks.cephadm:Teardown begin 2023-12-12T10:09:31.640 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-12T10:09:31.640 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T10:09:31.783 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:31 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3073209566' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-12T10:09:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.808+0000 7f28569c8700 1 -- 172.21.15.42:0/927242157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2850101dd0 msgr2=0x7f285010e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.808+0000 7f28569c8700 1 --2- 172.21.15.42:0/927242157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2850101dd0 0x7f285010e2e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f284800b0a0 tx=0x7f284800b3b0 comp rx=0 tx=0).stop 2023-12-12T10:09:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.808+0000 7f28569c8700 1 -- 172.21.15.42:0/927242157 shutdown_connections 2023-12-12T10:09:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.808+0000 7f28569c8700 1 --2- 172.21.15.42:0/927242157 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2850101dd0 0x7f285010e2e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:31.811 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.808+0000 7f28569c8700 1 --2- 172.21.15.42:0/927242157 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f28501074b0 0x7f2850107890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.808+0000 7f28569c8700 1 --2- 172.21.15.42:0/927242157 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2850101410 0x7f2850101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.808+0000 7f28569c8700 1 -- 172.21.15.42:0/927242157 >> 172.21.15.42:0/927242157 conn(0x7f28500fcf20 msgr2=0x7f28500ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.808+0000 7f28569c8700 1 -- 172.21.15.42:0/927242157 shutdown_connections 2023-12-12T10:09:31.812 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.808+0000 7f28569c8700 1 -- 172.21.15.42:0/927242157 wait complete. 2023-12-12T10:09:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f28569c8700 1 Processor -- start 2023-12-12T10:09:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f28569c8700 1 -- start start 2023-12-12T10:09:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f28569c8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2850101410 0x7f285019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:31.814 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f28569c8700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2850101dd0 0x7f285019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f28569c8700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28501074b0 0x7f28501976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f28569c8700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850197d00 con 0x7f28501074b0 2023-12-12T10:09:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f28569c8700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850197e70 con 0x7f2850101dd0 2023-12-12T10:09:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f28569c8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2850198030 con 0x7f2850101410 2023-12-12T10:09:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f2854f65700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28501074b0 0x7f28501976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f2854f65700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28501074b0 0x7f28501976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37166/0 (socket says 172.21.15.42:37166) 2023-12-12T10:09:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f2854f65700 1 -- 172.21.15.42:0/2607749690 learned_addr learned my addr 172.21.15.42:0/2607749690 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f284f7fe700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2850101dd0 0x7f285019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:31.815 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f284ffff700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2850101410 0x7f285019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f2854f65700 1 -- 172.21.15.42:0/2607749690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2850101410 msgr2=0x7f285019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f2854f65700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2850101410 0x7f285019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f2854f65700 1 -- 172.21.15.42:0/2607749690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2850101dd0 msgr2=0x7f285019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f2854f65700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2850101dd0 0x7f285019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.809+0000 7f2854f65700 1 -- 172.21.15.42:0/2607749690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2848009d00 con 0x7f28501074b0 2023-12-12T10:09:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.810+0000 7f2854f65700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28501074b0 0x7f28501976a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f284800b070 tx=0x7f2848008ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.810+0000 7f284d7fa700 1 -- 172.21.15.42:0/2607749690 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284800e040 con 0x7f28501074b0 2023-12-12T10:09:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.810+0000 7f284d7fa700 1 -- 172.21.15.42:0/2607749690 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f284801d470 con 0x7f28501074b0 2023-12-12T10:09:31.816 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.810+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28501982b0 con 0x7f28501074b0 2023-12-12T10:09:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.810+0000 7f284d7fa700 1 -- 172.21.15.42:0/2607749690 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28480052a0 con 0x7f28501074b0 2023-12-12T10:09:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.810+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28501a4560 con 0x7f28501074b0 2023-12-12T10:09:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.811+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2850109a30 con 0x7f28501074b0 2023-12-12T10:09:31.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.812+0000 7f284d7fa700 1 -- 172.21.15.42:0/2607749690 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f284801d5e0 con 0x7f28501074b0 2023-12-12T10:09:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.812+0000 7f284d7fa700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f28380a15f0 0x7f28380a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.813+0000 7f284d7fa700 1 -- 172.21.15.42:0/2607749690 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f28480cea40 con 0x7f28501074b0 2023-12-12T10:09:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.813+0000 7f284ffff700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f28380a15f0 0x7f28380a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.813+0000 7f284ffff700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f28380a15f0 0x7f28380a3ab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2840009be0 tx=0x7f2840009260 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:31.818 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:31.816+0000 7f284d7fa700 1 -- 172.21.15.42:0/2607749690 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f28480979f0 con 0x7f28501074b0 2023-12-12T10:09:31.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:31 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3073209566' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-12T10:09:31.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:31 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3073209566' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-12T10:09:32.064 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.062+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2850102b50 con 0x7f28501074b0 2023-12-12T10:09:32.065 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.062+0000 7f284d7fa700 1 -- 172.21.15.42:0/2607749690 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+11381 (secure 0 0 0) 0x7f2848024350 con 0x7f28501074b0 2023-12-12T10:09:32.065 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T10:09:32.066 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T10:08:25.272787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24487,"mds_3":24482,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T10:09:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f28380a15f0 msgr2=0x7f28380a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f28380a15f0 0x7f28380a3ab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2840009be0 tx=0x7f2840009260 comp rx=0 tx=0).stop 2023-12-12T10:09:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28501074b0 msgr2=0x7f28501976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28501074b0 0x7f28501976a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f284800b070 tx=0x7f2848008ff0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 shutdown_connections 2023-12-12T10:09:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f28501074b0 0x7f28501976a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2850101410 0x7f285019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.068 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f28380a15f0 0x7f28380a3ab0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 --2- 172.21.15.42:0/2607749690 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2850101dd0 0x7f285019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 >> 172.21.15.42:0/2607749690 conn(0x7f28500fcf20 msgr2=0x7f28500fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 shutdown_connections 2023-12-12T10:09:32.069 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.065+0000 7f28569c8700 1 -- 172.21.15.42:0/2607749690 wait complete. 2023-12-12T10:09:32.070 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 51 2023-12-12T10:09:32.081 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-12T10:09:32.210 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 -- 172.21.15.42:0/3299246282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760100590 msgr2=0x7fa760100970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3299246282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760100590 0x7fa760100970 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009990 tx=0x7fa75c009ca0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 -- 172.21.15.42:0/3299246282 shutdown_connections 2023-12-12T10:09:32.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3299246282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7601056b0 0x7fa76010d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3299246282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760100590 0x7fa760100970 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3299246282 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa760100f40 0x7fa760104f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 -- 172.21.15.42:0/3299246282 >> 172.21.15.42:0/3299246282 conn(0x7fa7600fbe50 msgr2=0x7fa7600fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:32.211 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 -- 172.21.15.42:0/3299246282 shutdown_connections 2023-12-12T10:09:32.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 -- 172.21.15.42:0/3299246282 wait complete. 2023-12-12T10:09:32.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.206+0000 7fa767ff9700 1 Processor -- start 2023-12-12T10:09:32.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa767ff9700 1 -- start start 2023-12-12T10:09:32.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa767ff9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa760100590 0x7fa76019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa767ff9700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7601056b0 0x7fa76019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa767ff9700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760197720 0x7fa760197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa767ff9700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa760198220 con 0x7fa760197720 2023-12-12T10:09:32.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa767ff9700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa760198390 con 0x7fa7601056b0 2023-12-12T10:09:32.212 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa767ff9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa760198550 con 0x7fa760100590 2023-12-12T10:09:32.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa766596700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760197720 0x7fa760197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa766596700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760197720 0x7fa760197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37178/0 (socket says 172.21.15.42:37178) 2023-12-12T10:09:32.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa766596700 1 -- 172.21.15.42:0/3531664608 learned_addr learned my addr 172.21.15.42:0/3531664608 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:32.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa765d95700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa760100590 0x7fa76019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa765594700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7601056b0 0x7fa76019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa766596700 1 -- 172.21.15.42:0/3531664608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa760100590 msgr2=0x7fa76019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa766596700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa760100590 0x7fa76019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa766596700 1 -- 172.21.15.42:0/3531664608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7601056b0 msgr2=0x7fa76019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.213 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa766596700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7601056b0 0x7fa76019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.214 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa766596700 1 -- 172.21.15.42:0/3531664608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c0095f0 con 0x7fa760197720 2023-12-12T10:09:32.214 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.207+0000 7fa766596700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760197720 0x7fa760197bd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa75800ef30 tx=0x7fa75800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:32.214 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.208+0000 7fa756ffd700 1 -- 172.21.15.42:0/3531664608 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa758018070 con 0x7fa760197720 2023-12-12T10:09:32.214 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.208+0000 7fa756ffd700 1 -- 172.21.15.42:0/3531664608 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa75800f040 con 0x7fa760197720 2023-12-12T10:09:32.214 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.208+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7601a44e0 con 0x7fa760197720 2023-12-12T10:09:32.214 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.208+0000 7fa756ffd700 1 -- 172.21.15.42:0/3531664608 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75800cd00 con 0x7fa760197720 2023-12-12T10:09:32.214 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.208+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7601a49a0 con 0x7fa760197720 2023-12-12T10:09:32.215 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.209+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa760109930 con 0x7fa760197720 2023-12-12T10:09:32.215 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.211+0000 7fa756ffd700 1 -- 172.21.15.42:0/3531664608 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fa758003a00 con 0x7fa760197720 2023-12-12T10:09:32.215 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.211+0000 7fa756ffd700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa74c0a15f0 0x7fa74c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.216 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.211+0000 7fa765d95700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa74c0a15f0 0x7fa74c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.216 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.211+0000 7fa756ffd700 1 -- 172.21.15.42:0/3531664608 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa7580cda10 con 0x7fa760197720 2023-12-12T10:09:32.216 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.212+0000 7fa765d95700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa74c0a15f0 0x7fa74c0a3ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00b2e0 tx=0x7fa75c005960 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:32.217 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.215+0000 7fa756ffd700 1 -- 172.21.15.42:0/3531664608 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fa758096960 con 0x7fa760197720 2023-12-12T10:09:32.466 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.463+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa760198c00 con 0x7fa760197720 2023-12-12T10:09:32.467 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.464+0000 7fa756ffd700 1 -- 172.21.15.42:0/3531664608 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+11381 (secure 0 0 0) 0x7fa758099f80 con 0x7fa760197720 2023-12-12T10:09:32.467 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T10:09:32.467 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/4073397880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":4073397880},{"type":"v1","addr":"172.21.15.42:6835","nonce":4073397880}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14559,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3743179195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3743179195},{"type":"v1","addr":"172.21.15.42:6837","nonce":3743179195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-12T09:56:37.902338+0000","modified":"2023-12-12T10:08:25.272787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24524,"mds_1":24508,"mds_2":24487,"mds_3":24482,"mds_4":24475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14616":{"gid":14616,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6839/229824980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":229824980},{"type":"v1","addr":"172.21.15.42:6839","nonce":229824980}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14628":{"gid":14628,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.42:6841/4221329195","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4221329195},{"type":"v1","addr":"172.21.15.42:6841","nonce":4221329195}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6833/1539682179","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":1539682179},{"type":"v1","addr":"172.21.15.111:6833","nonce":1539682179}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24430":{"gid":24430,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.173:6833/2859393185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6832","nonce":2859393185},{"type":"v1","addr":"172.21.15.173:6833","nonce":2859393185}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24452":{"gid":24452,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.111:6835/4127941301","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":4127941301},{"type":"v1","addr":"172.21.15.111:6835","nonce":4127941301}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24475":{"gid":24475,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.173:6835/3087189893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3087189893},{"type":"v1","addr":"172.21.15.173:6835","nonce":3087189893}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"h","rank":3,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.111:6837/3946741228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3946741228},{"type":"v1","addr":"172.21.15.111:6837","nonce":3946741228}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.173:6837/1667746348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1667746348},{"type":"v1","addr":"172.21.15.173:6837","nonce":1667746348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24508":{"gid":24508,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.173:6839/2977372273","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6838","nonce":2977372273},{"type":"v1","addr":"172.21.15.173:6839","nonce":2977372273}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.111:6839/2430709127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6838","nonce":2430709127},{"type":"v1","addr":"172.21.15.111:6839","nonce":2430709127}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-12T10:09:32.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa74c0a15f0 msgr2=0x7fa74c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa74c0a15f0 0x7fa74c0a3ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00b2e0 tx=0x7fa75c005960 comp rx=0 tx=0).stop 2023-12-12T10:09:32.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760197720 msgr2=0x7fa760197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760197720 0x7fa760197bd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa75800ef30 tx=0x7fa75800c3a0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 shutdown_connections 2023-12-12T10:09:32.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa760197720 0x7fa760197bd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.469 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa760100590 0x7fa76019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fa74c0a15f0 0x7fa74c0a3ab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 --2- 172.21.15.42:0/3531664608 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa7601056b0 0x7fa76019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 >> 172.21.15.42:0/3531664608 conn(0x7fa7600fbe50 msgr2=0x7fa760069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:32.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 shutdown_connections 2023-12-12T10:09:32.470 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.466+0000 7fa767ff9700 1 -- 172.21.15.42:0/3531664608 wait complete. 2023-12-12T10:09:32.470 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 51 2023-12-12T10:09:32.481 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-12T10:09:32.610 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 -- 172.21.15.42:0/647306638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494100580 msgr2=0x7f7494100960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 --2- 172.21.15.42:0/647306638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494100580 0x7f7494100960 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7490009990 tx=0x7f7490009ca0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 -- 172.21.15.42:0/647306638 shutdown_connections 2023-12-12T10:09:32.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 --2- 172.21.15.42:0/647306638 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74941056a0 0x7f749410d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 --2- 172.21.15.42:0/647306638 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494100580 0x7f7494100960 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 --2- 172.21.15.42:0/647306638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7494100f30 0x7f7494104f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 -- 172.21.15.42:0/647306638 >> 172.21.15.42:0/647306638 conn(0x7f74940fbe40 msgr2=0x7f74940fe260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:32.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 -- 172.21.15.42:0/647306638 shutdown_connections 2023-12-12T10:09:32.611 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 -- 172.21.15.42:0/647306638 wait complete. 2023-12-12T10:09:32.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.606+0000 7f749c1b7700 1 Processor -- start 2023-12-12T10:09:32.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749c1b7700 1 -- start start 2023-12-12T10:09:32.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749c1b7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7494100580 0x7f7494198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749c1b7700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74941056a0 0x7f7494198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749c1b7700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494199290 0x7f74941a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749c1b7700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74941a3fa0 con 0x7f7494199290 2023-12-12T10:09:32.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749c1b7700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74941a4110 con 0x7f74941056a0 2023-12-12T10:09:32.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749c1b7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74941a42d0 con 0x7f7494100580 2023-12-12T10:09:32.612 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749a754700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494199290 0x7f74941a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749a754700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494199290 0x7f74941a3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37192/0 (socket says 172.21.15.42:37192) 2023-12-12T10:09:32.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749a754700 1 -- 172.21.15.42:0/2596137055 learned_addr learned my addr 172.21.15.42:0/2596137055 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:32.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f7499f53700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7494100580 0x7f7494198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f7499752700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74941056a0 0x7f7494198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749a754700 1 -- 172.21.15.42:0/2596137055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7494100580 msgr2=0x7f7494198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749a754700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7494100580 0x7f7494198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749a754700 1 -- 172.21.15.42:0/2596137055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74941056a0 msgr2=0x7f7494198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749a754700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74941056a0 0x7f7494198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.613 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749a754700 1 -- 172.21.15.42:0/2596137055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74900095f0 con 0x7f7494199290 2023-12-12T10:09:32.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f749a754700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494199290 0x7f74941a3890 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f748c007ee0 tx=0x7f748c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:32.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.607+0000 7f748affd700 1 -- 172.21.15.42:0/2596137055 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00dd20 con 0x7f7494199290 2023-12-12T10:09:32.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.608+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74941a45b0 con 0x7f7494199290 2023-12-12T10:09:32.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.608+0000 7f748affd700 1 -- 172.21.15.42:0/2596137055 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748c00f040 con 0x7f7494199290 2023-12-12T10:09:32.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.608+0000 7f748affd700 1 -- 172.21.15.42:0/2596137055 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c0048a0 con 0x7f7494199290 2023-12-12T10:09:32.614 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.608+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74941a4a70 con 0x7f7494199290 2023-12-12T10:09:32.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.609+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74941088c0 con 0x7f7494199290 2023-12-12T10:09:32.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.610+0000 7f748affd700 1 -- 172.21.15.42:0/2596137055 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f748c003a00 con 0x7f7494199290 2023-12-12T10:09:32.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.610+0000 7f748affd700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f74800a15f0 0x7f74800a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.615 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.611+0000 7f748affd700 1 -- 172.21.15.42:0/2596137055 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f748c0cd530 con 0x7f7494199290 2023-12-12T10:09:32.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.611+0000 7f7499f53700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f74800a15f0 0x7f74800a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.611+0000 7f7499f53700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f74800a15f0 0x7f74800a3ab0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f749000b5c0 tx=0x7f7490005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:32.616 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.614+0000 7f748affd700 1 -- 172.21.15.42:0/2596137055 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f748c096450 con 0x7f7494199290 2023-12-12T10:09:32.817 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.815+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7494101a50 con 0x7f7494199290 2023-12-12T10:09:32.819 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.816+0000 7f748affd700 1 -- 172.21.15.42:0/2596137055 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+25310 (secure 0 0 0) 0x7f748c023090 con 0x7f7494199290 2023-12-12T10:09:32.819 INFO:teuthology.orchestra.run.smithi042.stdout: 2023-12-12T10:09:32.820 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":99,"fsid":"968ccaec-98d3-11ee-95a3-87774f69a715","created":"2023-12-12T09:49:23.808107+0000","modified":"2023-12-12T10:09:07.830289+0000","last_up_change":"2023-12-12T09:55:55.973916+0000","last_in_change":"2023-12-12T09:55:38.178707+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-12T09:52:42.268062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-12T09:56:35.884237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-12T09:56:37.283813+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-12T09:56:40.918865+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"33717319-c6c1-4217-91a8-02927794dbce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6803","nonce":4116302283}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6805","nonce":4116302283}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6809","nonce":4116302283}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":4116302283},{"type":"v1","addr":"172.21.15.42:6807","nonce":4116302283}]},"public_addr":"172.21.15.42:6803/4116302283","cluster_addr":"172.21.15.42:6805/4116302283","heartbeat_back_addr":"172.21.15.42:6809/4116302283","heartbeat_front_addr":"172.21.15.42:6807/4116302283","state":["exists","up"]},{"osd":1,"uuid":"067bc3ee-1118-46a9-beb0-cd9e9a237c25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6801","nonce":804818401}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6803","nonce":804818401}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6807","nonce":804818401}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":804818401},{"type":"v1","addr":"172.21.15.111:6805","nonce":804818401}]},"public_addr":"172.21.15.111:6801/804818401","cluster_addr":"172.21.15.111:6803/804818401","heartbeat_back_addr":"172.21.15.111:6807/804818401","heartbeat_front_addr":"172.21.15.111:6805/804818401","state":["exists","up"]},{"osd":2,"uuid":"1d209df1-c616-4f0e-8a58-5b0c1839967c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6801","nonce":3785124415}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6803","nonce":3785124415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6807","nonce":3785124415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":3785124415},{"type":"v1","addr":"172.21.15.173:6805","nonce":3785124415}]},"public_addr":"172.21.15.173:6801/3785124415","cluster_addr":"172.21.15.173:6803/3785124415","heartbeat_back_addr":"172.21.15.173:6807/3785124415","heartbeat_front_addr":"172.21.15.173:6805/3785124415","state":["exists","up"]},{"osd":3,"uuid":"2c3fe771-613f-4d10-956b-78db77d66068","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6811","nonce":1515738461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6813","nonce":1515738461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6817","nonce":1515738461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":1515738461},{"type":"v1","addr":"172.21.15.42:6815","nonce":1515738461}]},"public_addr":"172.21.15.42:6811/1515738461","cluster_addr":"172.21.15.42:6813/1515738461","heartbeat_back_addr":"172.21.15.42:6817/1515738461","heartbeat_front_addr":"172.21.15.42:6815/1515738461","state":["exists","up"]},{"osd":4,"uuid":"664d9eb1-f016-4943-a329-e18c4a7d9377","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6809","nonce":724947563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6811","nonce":724947563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6815","nonce":724947563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":724947563},{"type":"v1","addr":"172.21.15.111:6813","nonce":724947563}]},"public_addr":"172.21.15.111:6809/724947563","cluster_addr":"172.21.15.111:6811/724947563","heartbeat_back_addr":"172.21.15.111:6815/724947563","heartbeat_front_addr":"172.21.15.111:6813/724947563","state":["exists","up"]},{"osd":5,"uuid":"5d8ef1de-ecd3-4b7a-a49e-66e5ad4cbc13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6809","nonce":3342915848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6811","nonce":3342915848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6815","nonce":3342915848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":3342915848},{"type":"v1","addr":"172.21.15.173:6813","nonce":3342915848}]},"public_addr":"172.21.15.173:6809/3342915848","cluster_addr":"172.21.15.173:6811/3342915848","heartbeat_back_addr":"172.21.15.173:6815/3342915848","heartbeat_front_addr":"172.21.15.173:6813/3342915848","state":["exists","up"]},{"osd":6,"uuid":"f1cf8ac5-f733-425d-9148-02d06c5871e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6819","nonce":288642128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6821","nonce":288642128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6825","nonce":288642128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":288642128},{"type":"v1","addr":"172.21.15.42:6823","nonce":288642128}]},"public_addr":"172.21.15.42:6819/288642128","cluster_addr":"172.21.15.42:6821/288642128","heartbeat_back_addr":"172.21.15.42:6825/288642128","heartbeat_front_addr":"172.21.15.42:6823/288642128","state":["exists","up"]},{"osd":7,"uuid":"a8c52f12-79ac-43b9-81af-d02bb17b812f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":50,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6817","nonce":1072744652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6819","nonce":1072744652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6823","nonce":1072744652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":1072744652},{"type":"v1","addr":"172.21.15.111:6821","nonce":1072744652}]},"public_addr":"172.21.15.111:6817/1072744652","cluster_addr":"172.21.15.111:6819/1072744652","heartbeat_back_addr":"172.21.15.111:6823/1072744652","heartbeat_front_addr":"172.21.15.111:6821/1072744652","state":["exists","up"]},{"osd":8,"uuid":"7e1e8185-cda2-4a3c-9ff2-fc2480152394","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6816","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6817","nonce":1898117914}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6819","nonce":1898117914}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6822","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6823","nonce":1898117914}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":1898117914},{"type":"v1","addr":"172.21.15.173:6821","nonce":1898117914}]},"public_addr":"172.21.15.173:6817/1898117914","cluster_addr":"172.21.15.173:6819/1898117914","heartbeat_back_addr":"172.21.15.173:6823/1898117914","heartbeat_front_addr":"172.21.15.173:6821/1898117914","state":["exists","up"]},{"osd":9,"uuid":"1c936a54-8440-4cbd-ba0b-79d236e99180","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6827","nonce":1755856143}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6829","nonce":1755856143}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6833","nonce":1755856143}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":1755856143},{"type":"v1","addr":"172.21.15.42:6831","nonce":1755856143}]},"public_addr":"172.21.15.42:6827/1755856143","cluster_addr":"172.21.15.42:6829/1755856143","heartbeat_back_addr":"172.21.15.42:6833/1755856143","heartbeat_front_addr":"172.21.15.42:6831/1755856143","state":["exists","up"]},{"osd":10,"uuid":"963a1538-8ad6-4832-9f32-cc7bb0642555","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6825","nonce":280900835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6827","nonce":280900835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6831","nonce":280900835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":280900835},{"type":"v1","addr":"172.21.15.111:6829","nonce":280900835}]},"public_addr":"172.21.15.111:6825/280900835","cluster_addr":"172.21.15.111:6827/280900835","heartbeat_back_addr":"172.21.15.111:6831/280900835","heartbeat_front_addr":"172.21.15.111:6829/280900835","state":["exists","up"]},{"osd":11,"uuid":"840f024c-8e3a-4949-8e31-27f28d498d95","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":78,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6825","nonce":1783755959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6826","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6827","nonce":1783755959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6831","nonce":1783755959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6828","nonce":1783755959},{"type":"v1","addr":"172.21.15.173:6829","nonce":1783755959}]},"public_addr":"172.21.15.173:6825/1783755959","cluster_addr":"172.21.15.173:6827/1783755959","heartbeat_back_addr":"172.21.15.173:6831/1783755959","heartbeat_front_addr":"172.21.15.173:6829/1783755959","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:51:57.685154+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:19.008735+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:52:40.142140+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:02.819243+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:23.995406+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:53:45.379915+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:07.449364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:28.809855+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:54:50.093098+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:12.240392+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:33.064400+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-12T09:55:54.289010+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.a","mappings":[{"from":5,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:6801/2637927159":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/2112736882":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/3552425799":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/3987075096":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/467026676":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/2920147377":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6801/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6801/1433305616":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3856214912":"2023-12-13T09:49:58.206187+0000","172.21.15.42:6800/1525180467":"2023-12-13T09:50:27.727252+0000","172.21.15.42:0/1613862950":"2023-12-13T09:50:27.727252+0000","172.21.15.42:6800/2637927159":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/4273364865":"2023-12-13T09:49:58.206187+0000","172.21.15.42:0/3046151373":"2023-12-13T09:58:36.825079+0000","172.21.15.42:0/2212533654":"2023-12-13T09:49:58.206187+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-12T10:09:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f74800a15f0 msgr2=0x7f74800a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f74800a15f0 0x7f74800a3ab0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f749000b5c0 tx=0x7f7490005fd0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494199290 msgr2=0x7f74941a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.821 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494199290 0x7f74941a3890 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f748c007ee0 tx=0x7f748c00d3b0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 shutdown_connections 2023-12-12T10:09:32.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7494199290 0x7f74941a3890 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7494100580 0x7f7494198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f74800a15f0 0x7f74800a3ab0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 --2- 172.21.15.42:0/2596137055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f74941056a0 0x7f7494198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 >> 172.21.15.42:0/2596137055 conn(0x7f74940fbe40 msgr2=0x7f74940fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:32.822 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 shutdown_connections 2023-12-12T10:09:32.823 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.818+0000 7f749c1b7700 1 -- 172.21.15.42:0/2596137055 wait complete. 2023-12-12T10:09:32.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:32 smithi173 ceph-mon[103308]: pgmap v337: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 717 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-12-12T10:09:32.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:32 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2607749690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T10:09:32.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:32 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3531664608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T10:09:32.832 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:32 smithi042 ceph-mon[98894]: pgmap v337: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 717 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-12-12T10:09:32.832 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:32 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2607749690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T10:09:32.832 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:32 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3531664608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T10:09:32.835 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-12T10:09:32.963 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.960+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/1211010346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4100b70 msgr2=0x7fe7f4100ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.963 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.960+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/1211010346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4100b70 0x7fe7f4100ff0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009990 tx=0x7fe7dc009ca0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.960+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/1211010346 shutdown_connections 2023-12-12T10:09:32.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.960+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/1211010346 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f41064a0 0x7fe7f4106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.960+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/1211010346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f4101530 0x7fe7f410d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.960+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/1211010346 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4100b70 0x7fe7f4100ff0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.960+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/1211010346 >> 172.21.15.42:0/1211010346 conn(0x7fe7f4076a00 msgr2=0x7fe7f40fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:32.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.960+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/1211010346 shutdown_connections 2023-12-12T10:09:32.964 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.960+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/1211010346 wait complete. 2023-12-12T10:09:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.961+0000 7fe7f8eb5700 1 Processor -- start 2023-12-12T10:09:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.961+0000 7fe7f8eb5700 1 -- start start 2023-12-12T10:09:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.961+0000 7fe7f8eb5700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f4100b70 0x7fe7f41020a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.961+0000 7fe7f8eb5700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4101530 0x7fe7f41025e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.961+0000 7fe7f8eb5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f41064a0 0x7fe7f4102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.961+0000 7fe7f8eb5700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f41a5df0 con 0x7fe7f4101530 2023-12-12T10:09:32.967 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.961+0000 7fe7f8eb5700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f41a5f60 con 0x7fe7f4100b70 2023-12-12T10:09:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.961+0000 7fe7f8eb5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f41a6120 con 0x7fe7f41064a0 2023-12-12T10:09:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.961+0000 7fe7f1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4101530 0x7fe7f41025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f1d9b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4101530 0x7fe7f41025e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37204/0 (socket says 172.21.15.42:37204) 2023-12-12T10:09:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f1d9b700 1 -- 172.21.15.42:0/2394005004 learned_addr learned my addr 172.21.15.42:0/2394005004 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f259c700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f4100b70 0x7fe7f41020a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f2d9d700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f41064a0 0x7fe7f4102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f1d9b700 1 -- 172.21.15.42:0/2394005004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f41064a0 msgr2=0x7fe7f4102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f1d9b700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f41064a0 0x7fe7f4102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.968 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f1d9b700 1 -- 172.21.15.42:0/2394005004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f4100b70 msgr2=0x7fe7f41020a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f1d9b700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f4100b70 0x7fe7f41020a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f1d9b700 1 -- 172.21.15.42:0/2394005004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc0095f0 con 0x7fe7f4101530 2023-12-12T10:09:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f1d9b700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4101530 0x7fe7f41025e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0098e0 tx=0x7fe7dc00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7eb7fe700 1 -- 172.21.15.42:0/2394005004 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc01d070 con 0x7fe7f4101530 2023-12-12T10:09:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7eb7fe700 1 -- 172.21.15.42:0/2394005004 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe7dc022470 con 0x7fe7f4101530 2023-12-12T10:09:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f41a63a0 con 0x7fe7f4101530 2023-12-12T10:09:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7eb7fe700 1 -- 172.21.15.42:0/2394005004 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc017650 con 0x7fe7f4101530 2023-12-12T10:09:32.969 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.962+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f41a6880 con 0x7fe7f4101530 2023-12-12T10:09:32.970 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.963+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f4101b80 con 0x7fe7f4101530 2023-12-12T10:09:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.965+0000 7fe7eb7fe700 1 -- 172.21.15.42:0/2394005004 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7fe7dc0225e0 con 0x7fe7f4101530 2023-12-12T10:09:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.965+0000 7fe7eb7fe700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe7e00a15f0 0x7fe7e00a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.965+0000 7fe7eb7fe700 1 -- 172.21.15.42:0/2394005004 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe7dc0ce8b0 con 0x7fe7f4101530 2023-12-12T10:09:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.965+0000 7fe7f259c700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe7e00a15f0 0x7fe7e00a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.966+0000 7fe7f259c700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe7e00a15f0 0x7fe7e00a3ab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005950 tx=0x7fe7e400b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:32.971 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:32.968+0000 7fe7eb7fe700 1 -- 172.21.15.42:0/2394005004 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7fe7dc097860 con 0x7fe7f4101530 2023-12-12T10:09:32.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:32 smithi111 ceph-mon[102565]: pgmap v337: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 717 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-12-12T10:09:32.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:32 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2607749690' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T10:09:32.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:32 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3531664608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-12T10:09:33.225 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.222+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe7f4067a80 con 0x7fe7f4101530 2023-12-12T10:09:33.227 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.224+0000 7fe7eb7fe700 1 -- 172.21.15.42:0/2394005004 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe7dc09ae80 con 0x7fe7f4101530 2023-12-12T10:09:33.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe7e00a15f0 msgr2=0x7fe7e00a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:33.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe7e00a15f0 0x7fe7e00a3ab0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4005950 tx=0x7fe7e400b410 comp rx=0 tx=0).stop 2023-12-12T10:09:33.232 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4101530 msgr2=0x7fe7f41025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:33.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4101530 0x7fe7f41025e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0098e0 tx=0x7fe7dc00f670 comp rx=0 tx=0).stop 2023-12-12T10:09:33.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 shutdown_connections 2023-12-12T10:09:33.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7fe7e00a15f0 0x7fe7e00a3ab0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7f4100b70 0x7fe7f41020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7f41064a0 0x7fe7f4102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 --2- 172.21.15.42:0/2394005004 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7f4101530 0x7fe7f41025e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 >> 172.21.15.42:0/2394005004 conn(0x7fe7f4076a00 msgr2=0x7fe7f40fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:33.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 shutdown_connections 2023-12-12T10:09:33.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.230+0000 7fe7f8eb5700 1 -- 172.21.15.42:0/2394005004 wait complete. 2023-12-12T10:09:33.235 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-12T10:09:33.246 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-12T10:09:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 -- 172.21.15.42:0/2930383827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 msgr2=0x7f764c1079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 --2- 172.21.15.42:0/2930383827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 0x7f764c1079e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f76480099c0 tx=0x7f7648009cd0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 -- 172.21.15.42:0/2930383827 shutdown_connections 2023-12-12T10:09:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 --2- 172.21.15.42:0/2930383827 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f764c101f00 0x7f764c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 --2- 172.21.15.42:0/2930383827 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 0x7f764c1079e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.392 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 --2- 172.21.15.42:0/2930383827 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f764c101540 0x7f764c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 -- 172.21.15.42:0/2930383827 >> 172.21.15.42:0/2930383827 conn(0x7f764c0fd010 msgr2=0x7f764c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 -- 172.21.15.42:0/2930383827 shutdown_connections 2023-12-12T10:09:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 -- 172.21.15.42:0/2930383827 wait complete. 2023-12-12T10:09:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.381+0000 7f76533c3700 1 Processor -- start 2023-12-12T10:09:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f76533c3700 1 -- start start 2023-12-12T10:09:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f76533c3700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f764c101540 0x7f764c19d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f76533c3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f764c101f00 0x7f764c19db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f76533c3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 0x7f764c197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:33.393 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f76533c3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c197da0 con 0x7f764c107600 2023-12-12T10:09:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f76533c3700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c197f10 con 0x7f764c101540 2023-12-12T10:09:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f76533c3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c1980d0 con 0x7f764c101f00 2023-12-12T10:09:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f7651960700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 0x7f764c197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f7651960700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 0x7f764c197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37244/0 (socket says 172.21.15.42:37244) 2023-12-12T10:09:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f7651960700 1 -- 172.21.15.42:0/445925853 learned_addr learned my addr 172.21.15.42:0/445925853 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f765115f700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f764c101540 0x7f764c19d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f765095e700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f764c101f00 0x7f764c19db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f7651960700 1 -- 172.21.15.42:0/445925853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f764c101f00 msgr2=0x7f764c19db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:33.394 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f7651960700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f764c101f00 0x7f764c19db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f7651960700 1 -- 172.21.15.42:0/445925853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f764c101540 msgr2=0x7f764c19d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f7651960700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f764c101540 0x7f764c19d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f7651960700 1 -- 172.21.15.42:0/445925853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480095f0 con 0x7f764c107600 2023-12-12T10:09:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f765095e700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f764c101f00 0x7f764c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:09:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f7651960700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 0x7f764c197740 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7644007ee0 tx=0x7f764400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.382+0000 7f76427fc700 1 -- 172.21.15.42:0/445925853 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644017070 con 0x7f764c107600 2023-12-12T10:09:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.383+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c1983b0 con 0x7f764c107600 2023-12-12T10:09:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.383+0000 7f76427fc700 1 -- 172.21.15.42:0/445925853 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f764400f040 con 0x7f764c107600 2023-12-12T10:09:33.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.383+0000 7f76427fc700 1 -- 172.21.15.42:0/445925853 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7644005740 con 0x7f764c107600 2023-12-12T10:09:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.383+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c1a4600 con 0x7f764c107600 2023-12-12T10:09:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.383+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c109a80 con 0x7f764c107600 2023-12-12T10:09:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.385+0000 7f76427fc700 1 -- 172.21.15.42:0/445925853 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f764400dc60 con 0x7f764c107600 2023-12-12T10:09:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.386+0000 7f76427fc700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f76380a15f0 0x7f76380a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:33.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.386+0000 7f765115f700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f76380a15f0 0x7f76380a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:33.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.386+0000 7f76427fc700 1 -- 172.21.15.42:0/445925853 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9883+0+0 (secure 0 0 0) 0x7f76440cd550 con 0x7f764c107600 2023-12-12T10:09:33.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.387+0000 7f765115f700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f76380a15f0 0x7f76380a3ab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7648009990 tx=0x7f7648003680 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:33.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.390+0000 7f76427fc700 1 -- 172.21.15.42:0/445925853 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f76440962c0 con 0x7f764c107600 2023-12-12T10:09:33.560 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-a[98870]: 2023-12-12T10:09:33.223+0000 7fc750c37700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-12T10:09:33.560 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-g[151122]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.560 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-g[151122]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.561 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mds[151146]: -- 172.21.15.42:0/3553576187 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556d7903b520 con 0x556d79011000 2023-12-12T10:09:33.561 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-g[151122]: starting mds.g at 2023-12-12T10:09:33.561 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-a[151506]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.561 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-a[151506]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.561 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mds[151530]: -- 172.21.15.42:0/3174972053 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b05d009520 con 0x55b05cfde800 2023-12-12T10:09:33.561 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-a[151506]: starting mds.a at 2023-12-12T10:09:33.562 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-e[138965]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.562 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-e[138965]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.562 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mds[138989]: -- 172.21.15.111:0/1325824904 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564987ce7520 con 0x564987cbcc00 2023-12-12T10:09:33.562 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-e[138965]: starting mds.e at 2023-12-12T10:09:33.563 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-i[140500]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.563 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-i[140500]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.563 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mds[140524]: -- 172.21.15.173:0/106767483 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cd401f3520 con 0x55cd401c9000 2023-12-12T10:09:33.563 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-i[140500]: starting mds.i at 2023-12-12T10:09:33.563 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.564 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.564 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mds[139673]: -- 172.21.15.111:0/4137235830 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55847ab13520 con 0x55847aae9000 2023-12-12T10:09:33.564 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: starting mds.b at 2023-12-12T10:09:33.564 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-c[140159]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.564 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-c[140159]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.564 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mds[140183]: -- 172.21.15.173:0/2267085743 <== mon.1 v2:172.21.15.173:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a22e753520 con 0x55a22e729000 2023-12-12T10:09:33.565 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-c[140159]: starting mds.c at 2023-12-12T10:09:33.565 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-k[138626]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.565 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-k[138626]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.565 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mds[138650]: -- 172.21.15.111:0/3191587630 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560de9709520 con 0x560de96dec00 2023-12-12T10:09:33.565 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-k[138626]: starting mds.k at 2023-12-12T10:09:33.566 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-f[139821]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.566 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-f[139821]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.566 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mds[139845]: -- 172.21.15.173:0/3278120623 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ce6f3c5520 con 0x55ce6f39a800 2023-12-12T10:09:33.566 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-f[139821]: starting mds.f at 2023-12-12T10:09:33.566 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-h[139308]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.566 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-h[139308]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.567 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mds[139333]: -- 172.21.15.111:0/4280895973 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5572f11c9520 con 0x5572f119ec00 2023-12-12T10:09:33.567 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-h[139308]: starting mds.h at 2023-12-12T10:09:33.567 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-l[139473]: ignoring --setuser ceph since I am not root 2023-12-12T10:09:33.567 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-l[139473]: ignoring --setgroup ceph since I am not root 2023-12-12T10:09:33.567 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mds[139497]: -- 172.21.15.173:0/1816647720 <== mon.2 v2:172.21.15.111:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8eb9d3520 con 0x55a8eb9a8c00 2023-12-12T10:09:33.567 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-l[139473]: starting mds.l at 2023-12-12T10:09:33.647 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:33.645+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f764c068340 con 0x7f764c107600 2023-12-12T10:09:33.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2596137055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T10:09:33.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2394005004' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-12T10:09:33.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mon[103308]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-12T10:09:33.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mon[103308]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-12T10:09:33.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/2394005004' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-12T10:09:33.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mon[103308]: osdmap e100: 12 total, 12 up, 12 in 2023-12-12T10:09:33.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mon[103308]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-12-12T10:09:33.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:33 smithi173 ceph-mon[103308]: osdmap e101: 12 total, 12 up, 12 in 2023-12-12T10:09:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2596137055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T10:09:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2394005004' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-12T10:09:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mon[98894]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-12T10:09:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mon[98894]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-12T10:09:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/2394005004' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-12T10:09:33.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mon[98894]: osdmap e100: 12 total, 12 up, 12 in 2023-12-12T10:09:33.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mon[98894]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-12-12T10:09:33.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:33 smithi042 ceph-mon[98894]: osdmap e101: 12 total, 12 up, 12 in 2023-12-12T10:09:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2596137055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-12T10:09:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2394005004' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-12T10:09:33.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mon[102565]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-12T10:09:33.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mon[102565]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-12T10:09:33.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/2394005004' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-12T10:09:33.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mon[102565]: osdmap e100: 12 total, 12 up, 12 in 2023-12-12T10:09:33.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mon[102565]: fsmap cephfs:0/5 2 up:standby, 5 failed 2023-12-12T10:09:33.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:33 smithi111 ceph-mon[102565]: osdmap e101: 12 total, 12 up, 12 in 2023-12-12T10:09:34.229 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.226+0000 7f76427fc700 1 -- 172.21.15.42:0/445925853 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 104+0+0 (secure 0 0 0) 0x7f76440998e0 con 0x7f764c107600 2023-12-12T10:09:34.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f76380a15f0 msgr2=0x7f76380a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:34.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f76380a15f0 0x7f76380a3ab0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7648009990 tx=0x7f7648003680 comp rx=0 tx=0).stop 2023-12-12T10:09:34.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 msgr2=0x7f764c197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:34.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 0x7f764c197740 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f7644007ee0 tx=0x7f764400d3b0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 shutdown_connections 2023-12-12T10:09:34.233 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f76380a15f0 0x7f76380a3ab0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f764c101540 0x7f764c19d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f764c107600 0x7f764c197740 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 --2- 172.21.15.42:0/445925853 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f764c101f00 0x7f764c19db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 >> 172.21.15.42:0/445925853 conn(0x7f764c0fd010 msgr2=0x7f764c0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 shutdown_connections 2023-12-12T10:09:34.234 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.230+0000 7f76533c3700 1 -- 172.21.15.42:0/445925853 wait complete. 2023-12-12T10:09:34.252 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-12T10:09:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.389+0000 7f9d30f7b700 1 -- 172.21.15.42:0/1893781858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 msgr2=0x7f9d2c100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:34.395 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.389+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/1893781858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 0x7f9d2c100880 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009990 tx=0x7f9d14009ca0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.389+0000 7f9d30f7b700 1 -- 172.21.15.42:0/1893781858 shutdown_connections 2023-12-12T10:09:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.389+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/1893781858 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d2c100dc0 0x7f9d2c10d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.389+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/1893781858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d2c106490 0x7f9d2c106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.389+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/1893781858 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 0x7f9d2c100880 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.389+0000 7f9d30f7b700 1 -- 172.21.15.42:0/1893781858 >> 172.21.15.42:0/1893781858 conn(0x7f9d2c077560 msgr2=0x7f9d2c077970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:34.396 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.389+0000 7f9d30f7b700 1 -- 172.21.15.42:0/1893781858 shutdown_connections 2023-12-12T10:09:34.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.389+0000 7f9d30f7b700 1 -- 172.21.15.42:0/1893781858 wait complete. 2023-12-12T10:09:34.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d30f7b700 1 Processor -- start 2023-12-12T10:09:34.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d30f7b700 1 -- start start 2023-12-12T10:09:34.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d30f7b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 0x7f9d2c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:34.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d30f7b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d2c100dc0 0x7f9d2c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:34.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d30f7b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d2c106490 0x7f9d2c1a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:34.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d30f7b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c199540 con 0x7f9d2c100400 2023-12-12T10:09:34.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d30f7b700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c1a3d50 con 0x7f9d2c100dc0 2023-12-12T10:09:34.397 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d30f7b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c1a3e90 con 0x7f9d2c106490 2023-12-12T10:09:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d2a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 0x7f9d2c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d2a59c700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 0x7f9d2c198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37258/0 (socket says 172.21.15.42:37258) 2023-12-12T10:09:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.390+0000 7f9d2a59c700 1 -- 172.21.15.42:0/3012327189 learned_addr learned my addr 172.21.15.42:0/3012327189 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d29d9b700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d2c100dc0 0x7f9d2c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d2ad9d700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d2c106490 0x7f9d2c1a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d2a59c700 1 -- 172.21.15.42:0/3012327189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d2c106490 msgr2=0x7f9d2c1a3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d2a59c700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d2c106490 0x7f9d2c1a3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d2a59c700 1 -- 172.21.15.42:0/3012327189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d2c100dc0 msgr2=0x7f9d2c198cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:34.398 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d2a59c700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d2c100dc0 0x7f9d2c198cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:34.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d2a59c700 1 -- 172.21.15.42:0/3012327189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d140095f0 con 0x7f9d2c100400 2023-12-12T10:09:34.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d2ad9d700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d2c106490 0x7f9d2c1a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:09:34.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d2a59c700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 0x7f9d2c198790 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00e9c0 tx=0x7f9d1c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:34.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d237fe700 1 -- 172.21.15.42:0/3012327189 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c0097b0 con 0x7f9d2c100400 2023-12-12T10:09:34.399 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c1a4120 con 0x7f9d2c100400 2023-12-12T10:09:34.400 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d237fe700 1 -- 172.21.15.42:0/3012327189 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d1c004cf0 con 0x7f9d2c100400 2023-12-12T10:09:34.401 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d237fe700 1 -- 172.21.15.42:0/3012327189 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c005600 con 0x7f9d2c100400 2023-12-12T10:09:34.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.391+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c1a45b0 con 0x7f9d2c100400 2023-12-12T10:09:34.402 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.392+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c108a10 con 0x7f9d2c100400 2023-12-12T10:09:34.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.395+0000 7f9d237fe700 1 -- 172.21.15.42:0/3012327189 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f9d1c00cc60 con 0x7f9d2c100400 2023-12-12T10:09:34.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.395+0000 7f9d237fe700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f9d180a1670 0x7f9d180a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:34.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.395+0000 7f9d237fe700 1 -- 172.21.15.42:0/3012327189 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9895+0+0 (secure 0 0 0) 0x7f9d1c014070 con 0x7f9d2c100400 2023-12-12T10:09:34.403 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.395+0000 7f9d29d9b700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f9d180a1670 0x7f9d180a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.396+0000 7f9d29d9b700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f9d180a1670 0x7f9d180a3b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400b2e0 tx=0x7f9d14005960 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:34.404 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.398+0000 7f9d237fe700 1 -- 172.21.15.42:0/3012327189 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f9d1c096840 con 0x7f9d2c100400 2023-12-12T10:09:34.605 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:34.602+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9d2c101b30 con 0x7f9d2c100400 2023-12-12T10:09:34.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: pgmap v338: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 717 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-12-12T10:09:34.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/445925853' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-12T10:09:34.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-12T10:09:34.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-12T10:09:34.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: Cluster is now healthy 2023-12-12T10:09:34.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/445925853' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-12T10:09:34.824 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6834/3418192318,v1:172.21.15.173:6835/3418192318] up:boot 2023-12-12T10:09:34.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6832/853300638,v1:172.21.15.111:6833/853300638] up:boot 2023-12-12T10:09:34.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6836/2270254759,v1:172.21.15.111:6837/2270254759] up:boot 2023-12-12T10:09:34.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6832/2736456527,v1:172.21.15.173:6833/2736456527] up:boot 2023-12-12T10:09:34.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6838/3563511502,v1:172.21.15.111:6839/3563511502] up:boot 2023-12-12T10:09:34.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.111:6834/1396780744,v1:172.21.15.111:6835/1396780744] up:boot 2023-12-12T10:09:34.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6836/2677613170,v1:172.21.15.173:6837/2677613170] up:boot 2023-12-12T10:09:34.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.173:6838/99092412,v1:172.21.15.173:6839/99092412] up:boot 2023-12-12T10:09:34.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.42:6839/3866490941,v1:172.21.15.42:6841/3866490941] up:boot 2023-12-12T10:09:34.825 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: mds.? [v2:172.21.15.42:6838/193598853,v1:172.21.15.42:6840/193598853] up:boot 2023-12-12T10:09:34.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: fsmap 12 up:standby 2023-12-12T10:09:34.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-12T10:09:34.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-12T10:09:34.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-12T10:09:34.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-12T10:09:34.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-12T10:09:34.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-12T10:09:34.826 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-12T10:09:34.827 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T10:09:34.827 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-12T10:09:34.827 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-12T10:09:34.827 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:34 smithi173 ceph-mon[103308]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T10:09:34.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: pgmap v338: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 717 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-12-12T10:09:34.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/445925853' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-12T10:09:34.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-12T10:09:34.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-12T10:09:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: Cluster is now healthy 2023-12-12T10:09:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/445925853' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-12T10:09:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6834/3418192318,v1:172.21.15.173:6835/3418192318] up:boot 2023-12-12T10:09:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6832/853300638,v1:172.21.15.111:6833/853300638] up:boot 2023-12-12T10:09:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6836/2270254759,v1:172.21.15.111:6837/2270254759] up:boot 2023-12-12T10:09:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6832/2736456527,v1:172.21.15.173:6833/2736456527] up:boot 2023-12-12T10:09:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6838/3563511502,v1:172.21.15.111:6839/3563511502] up:boot 2023-12-12T10:09:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.111:6834/1396780744,v1:172.21.15.111:6835/1396780744] up:boot 2023-12-12T10:09:34.905 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6836/2677613170,v1:172.21.15.173:6837/2677613170] up:boot 2023-12-12T10:09:34.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.173:6838/99092412,v1:172.21.15.173:6839/99092412] up:boot 2023-12-12T10:09:34.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.42:6839/3866490941,v1:172.21.15.42:6841/3866490941] up:boot 2023-12-12T10:09:34.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: mds.? [v2:172.21.15.42:6838/193598853,v1:172.21.15.42:6840/193598853] up:boot 2023-12-12T10:09:34.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: fsmap 12 up:standby 2023-12-12T10:09:34.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-12T10:09:34.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-12T10:09:34.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-12T10:09:34.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-12T10:09:34.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-12T10:09:34.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-12T10:09:34.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-12T10:09:34.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T10:09:34.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-12T10:09:34.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-12T10:09:34.907 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:34 smithi042 ceph-mon[98894]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T10:09:34.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: pgmap v338: 129 pgs: 4 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 118 active+clean; 717 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2023-12-12T10:09:34.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/445925853' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-12T10:09:34.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-12T10:09:34.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-12T10:09:34.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: Cluster is now healthy 2023-12-12T10:09:34.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/445925853' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-12T10:09:34.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6834/3418192318,v1:172.21.15.173:6835/3418192318] up:boot 2023-12-12T10:09:34.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6832/853300638,v1:172.21.15.111:6833/853300638] up:boot 2023-12-12T10:09:34.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6836/2270254759,v1:172.21.15.111:6837/2270254759] up:boot 2023-12-12T10:09:34.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6832/2736456527,v1:172.21.15.173:6833/2736456527] up:boot 2023-12-12T10:09:34.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6838/3563511502,v1:172.21.15.111:6839/3563511502] up:boot 2023-12-12T10:09:34.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.111:6834/1396780744,v1:172.21.15.111:6835/1396780744] up:boot 2023-12-12T10:09:35.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6836/2677613170,v1:172.21.15.173:6837/2677613170] up:boot 2023-12-12T10:09:35.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.173:6838/99092412,v1:172.21.15.173:6839/99092412] up:boot 2023-12-12T10:09:35.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.42:6839/3866490941,v1:172.21.15.42:6841/3866490941] up:boot 2023-12-12T10:09:35.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: mds.? [v2:172.21.15.42:6838/193598853,v1:172.21.15.42:6840/193598853] up:boot 2023-12-12T10:09:35.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: fsmap 12 up:standby 2023-12-12T10:09:35.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-12T10:09:35.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-12T10:09:35.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-12T10:09:35.000 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-12T10:09:35.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-12T10:09:35.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-12T10:09:35.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-12T10:09:35.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T10:09:35.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-12T10:09:35.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-12T10:09:35.001 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:34 smithi111 ceph-mon[102565]: from='mgr.15528 172.21.15.42:0/4185323194' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-12T10:09:35.574 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.571+0000 7f9d237fe700 1 -- 172.21.15.42:0/3012327189 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9d1c099e60 con 0x7f9d2c100400 2023-12-12T10:09:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f9d180a1670 msgr2=0x7f9d180a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f9d180a1670 0x7f9d180a3b30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400b2e0 tx=0x7f9d14005960 comp rx=0 tx=0).stop 2023-12-12T10:09:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 msgr2=0x7f9d2c198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:35.579 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 0x7f9d2c198790 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00e9c0 tx=0x7f9d1c00ecd0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.580 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 shutdown_connections 2023-12-12T10:09:35.580 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d2c106490 0x7f9d2c1a3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.580 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9d2c100400 0x7f9d2c198790 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f9d180a1670 0x7f9d180a3b30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 --2- 172.21.15.42:0/3012327189 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d2c100dc0 0x7f9d2c198cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.576+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 >> 172.21.15.42:0/3012327189 conn(0x7f9d2c077560 msgr2=0x7f9d2c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:35.581 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.577+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 shutdown_connections 2023-12-12T10:09:35.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.578+0000 7f9d30f7b700 1 -- 172.21.15.42:0/3012327189 wait complete. 2023-12-12T10:09:35.582 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2023-12-12T10:09:35.601 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-12T10:09:35.738 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.735+0000 7f7b99d77700 1 -- 172.21.15.42:0/226788382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b94108f70 msgr2=0x7f7b94116be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:35.739 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.735+0000 7f7b99d77700 1 --2- 172.21.15.42:0/226788382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b94108f70 0x7f7b94116be0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00b0a0 tx=0x7f7b8c00b3b0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.739 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.735+0000 7f7b99d77700 1 -- 172.21.15.42:0/226788382 shutdown_connections 2023-12-12T10:09:35.739 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.735+0000 7f7b99d77700 1 --2- 172.21.15.42:0/226788382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b94108f70 0x7f7b94116be0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.739 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.735+0000 7f7b99d77700 1 --2- 172.21.15.42:0/226788382 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b9410edc0 0x7f7b9410f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.739 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.735+0000 7f7b99d77700 1 --2- 172.21.15.42:0/226788382 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b941085d0 0x7f7b94108a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.739 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.735+0000 7f7b99d77700 1 -- 172.21.15.42:0/226788382 >> 172.21.15.42:0/226788382 conn(0x7f7b940721a0 msgr2=0x7f7b9406d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:35.739 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.735+0000 7f7b99d77700 1 -- 172.21.15.42:0/226788382 shutdown_connections 2023-12-12T10:09:35.739 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.735+0000 7f7b99d77700 1 -- 172.21.15.42:0/226788382 wait complete. 2023-12-12T10:09:35.740 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.736+0000 7f7b99d77700 1 Processor -- start 2023-12-12T10:09:35.742 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.736+0000 7f7b99d77700 1 -- start start 2023-12-12T10:09:35.742 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.736+0000 7f7b99d77700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b941085d0 0x7f7b941097b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:35.742 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.736+0000 7f7b99d77700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94108f70 0x7f7b94109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:35.742 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.736+0000 7f7b99d77700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b9410edc0 0x7f7b9410a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:35.742 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.736+0000 7f7b99d77700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b941ae650 con 0x7f7b9410edc0 2023-12-12T10:09:35.742 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.736+0000 7f7b99d77700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b941ae7c0 con 0x7f7b94108f70 2023-12-12T10:09:35.742 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.736+0000 7f7b99d77700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b941ae980 con 0x7f7b941085d0 2023-12-12T10:09:35.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b9410edc0 0x7f7b9410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:35.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b93fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b9410edc0 0x7f7b9410a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37276/0 (socket says 172.21.15.42:37276) 2023-12-12T10:09:35.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b93fff700 1 -- 172.21.15.42:0/395962287 learned_addr learned my addr 172.21.15.42:0/395962287 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:35.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b92ffd700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94108f70 0x7f7b94109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:35.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b937fe700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b941085d0 0x7f7b941097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:35.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b93fff700 1 -- 172.21.15.42:0/395962287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b941085d0 msgr2=0x7f7b941097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:35.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b93fff700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b941085d0 0x7f7b941097b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.743 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b93fff700 1 -- 172.21.15.42:0/395962287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94108f70 msgr2=0x7f7b94109cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b93fff700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94108f70 0x7f7b94109cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b93fff700 1 -- 172.21.15.42:0/395962287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c009d00 con 0x7f7b9410edc0 2023-12-12T10:09:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b93fff700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b9410edc0 0x7f7b9410a230 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0077f0 tx=0x7f7b8c007820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b90ff9700 1 -- 172.21.15.42:0/395962287 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c00e040 con 0x7f7b9410edc0 2023-12-12T10:09:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b90ff9700 1 -- 172.21.15.42:0/395962287 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b8c004c60 con 0x7f7b9410edc0 2023-12-12T10:09:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b941aec00 con 0x7f7b9410edc0 2023-12-12T10:09:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b90ff9700 1 -- 172.21.15.42:0/395962287 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0052a0 con 0x7f7b9410edc0 2023-12-12T10:09:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.737+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b941af110 con 0x7f7b9410edc0 2023-12-12T10:09:35.744 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.738+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b94112330 con 0x7f7b9410edc0 2023-12-12T10:09:35.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.739+0000 7f7b90ff9700 1 -- 172.21.15.42:0/395962287 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f7b8c01b450 con 0x7f7b9410edc0 2023-12-12T10:09:35.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.740+0000 7f7b90ff9700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7b7c0a1670 0x7f7b7c0a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:35.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.740+0000 7f7b90ff9700 1 -- 172.21.15.42:0/395962287 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 9313+0+0 (secure 0 0 0) 0x7f7b8c0ceb50 con 0x7f7b9410edc0 2023-12-12T10:09:35.745 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.740+0000 7f7b937fe700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7b7c0a1670 0x7f7b7c0a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:35.746 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.740+0000 7f7b937fe700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7b7c0a1670 0x7f7b7c0a3b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b94069440 tx=0x7f7b8400b410 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:35.747 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.744+0000 7f7b90ff9700 1 -- 172.21.15.42:0/395962287 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f7b8c097b80 con 0x7f7b9410edc0 2023-12-12T10:09:35.823 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:35 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3012327189' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-12T10:09:35.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:35 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3012327189' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-12T10:09:35.950 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:35.947+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b940696f0 con 0x7f7b9410edc0 2023-12-12T10:09:35.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:35 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3012327189' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-12T10:09:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.573+0000 7f7b90ff9700 1 -- 172.21.15.42:0/395962287 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7b8c09b1a0 con 0x7f7b9410edc0 2023-12-12T10:09:36.581 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.579+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7b7c0a1670 msgr2=0x7f7b7c0a3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:36.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.579+0000 7f7b99d77700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7b7c0a1670 0x7f7b7c0a3b30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b94069440 tx=0x7f7b8400b410 comp rx=0 tx=0).stop 2023-12-12T10:09:36.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.579+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b9410edc0 msgr2=0x7f7b9410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:36.582 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.579+0000 7f7b99d77700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b9410edc0 0x7f7b9410a230 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0077f0 tx=0x7f7b8c007820 comp rx=0 tx=0).stop 2023-12-12T10:09:36.584 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.582+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 shutdown_connections 2023-12-12T10:09:36.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.582+0000 7f7b99d77700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b9410edc0 0x7f7b9410a230 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.582+0000 7f7b99d77700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b941085d0 0x7f7b941097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.582+0000 7f7b99d77700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f7b7c0a1670 0x7f7b7c0a3b30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.582+0000 7f7b99d77700 1 --2- 172.21.15.42:0/395962287 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7b94108f70 0x7f7b94109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.582+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 >> 172.21.15.42:0/395962287 conn(0x7f7b940721a0 msgr2=0x7f7b9406b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:36.585 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.582+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 shutdown_connections 2023-12-12T10:09:36.586 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.582+0000 7f7b99d77700 1 -- 172.21.15.42:0/395962287 wait complete. 2023-12-12T10:09:36.587 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2023-12-12T10:09:36.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-12-12T10:09:36.757 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.754+0000 7f66f61bc700 1 -- 172.21.15.42:0/724470291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 msgr2=0x7f66f01008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.754+0000 7f66f61bc700 1 --2- 172.21.15.42:0/724470291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 0x7f66f01008a0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f66d8009990 tx=0x7f66d8009ca0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.754+0000 7f66f61bc700 1 -- 172.21.15.42:0/724470291 shutdown_connections 2023-12-12T10:09:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.754+0000 7f66f61bc700 1 --2- 172.21.15.42:0/724470291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f0100de0 0x7f66f010d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.754+0000 7f66f61bc700 1 --2- 172.21.15.42:0/724470291 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f01064c0 0x7f66f01068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.754+0000 7f66f61bc700 1 --2- 172.21.15.42:0/724470291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 0x7f66f01008a0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.754+0000 7f66f61bc700 1 -- 172.21.15.42:0/724470291 >> 172.21.15.42:0/724470291 conn(0x7f66f00775a0 msgr2=0x7f66f00779b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:36.758 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.754+0000 7f66f61bc700 1 -- 172.21.15.42:0/724470291 shutdown_connections 2023-12-12T10:09:36.759 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.754+0000 7f66f61bc700 1 -- 172.21.15.42:0/724470291 wait complete. 2023-12-12T10:09:36.759 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66f61bc700 1 Processor -- start 2023-12-12T10:09:36.759 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66f61bc700 1 -- start start 2023-12-12T10:09:36.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66f61bc700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 0x7f66f019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:36.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66f61bc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f0100de0 0x7f66f019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:36.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66f61bc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f01064c0 0x7f66f01977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:36.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66f61bc700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0197e60 con 0x7f66f0100420 2023-12-12T10:09:36.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66f61bc700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0197fd0 con 0x7f66f01064c0 2023-12-12T10:09:36.761 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66f61bc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0198190 con 0x7f66f0100de0 2023-12-12T10:09:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 0x7f66f019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66ef7fe700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 0x7f66f019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37286/0 (socket says 172.21.15.42:37286) 2023-12-12T10:09:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66ef7fe700 1 -- 172.21.15.42:0/3661036248 learned_addr learned my addr 172.21.15.42:0/3661036248 (peer_addr_for_me v2:172.21.15.42:0/0) 2023-12-12T10:09:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66effff700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f01064c0 0x7f66f01977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.755+0000 7f66eeffd700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f0100de0 0x7f66f019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66ef7fe700 1 -- 172.21.15.42:0/3661036248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f0100de0 msgr2=0x7f66f019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66ef7fe700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f0100de0 0x7f66f019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66ef7fe700 1 -- 172.21.15.42:0/3661036248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f01064c0 msgr2=0x7f66f01977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:36.762 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66ef7fe700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f01064c0 0x7f66f01977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66ef7fe700 1 -- 172.21.15.42:0/3661036248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d80095f0 con 0x7f66f0100420 2023-12-12T10:09:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66eeffd700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f0100de0 0x7f66f019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-12T10:09:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66ef7fe700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 0x7f66f019d6c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f66e000d980 tx=0x7f66e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66ecff9700 1 -- 172.21.15.42:0/3661036248 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0004220 con 0x7f66f0100420 2023-12-12T10:09:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66ecff9700 1 -- 172.21.15.42:0/3661036248 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66e0007640 con 0x7f66f0100420 2023-12-12T10:09:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f0198470 con 0x7f66f0100420 2023-12-12T10:09:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66ecff9700 1 -- 172.21.15.42:0/3661036248 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0005610 con 0x7f66f0100420 2023-12-12T10:09:36.763 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.756+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f01a46b0 con 0x7f66f0100420 2023-12-12T10:09:36.764 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.757+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f0109a50 con 0x7f66f0100420 2023-12-12T10:09:36.764 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.758+0000 7f66ecff9700 1 -- 172.21.15.42:0/3661036248 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 25) v1 ==== 136137+0+0 (secure 0 0 0) 0x7f66e0004750 con 0x7f66f0100420 2023-12-12T10:09:36.764 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.759+0000 7f66ecff9700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f66dc0a15f0 0x7f66dc0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-12T10:09:36.765 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.759+0000 7f66ecff9700 1 -- 172.21.15.42:0/3661036248 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8754+0+0 (secure 0 0 0) 0x7f66e00cd510 con 0x7f66f0100420 2023-12-12T10:09:36.765 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.759+0000 7f66eeffd700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f66dc0a15f0 0x7f66dc0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-12T10:09:36.765 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.759+0000 7f66eeffd700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f66dc0a15f0 0x7f66dc0a3ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66d800b2e0 tx=0x7f66d8005fd0 comp rx=0 tx=0).ready entity=mgr.15528 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-12T10:09:36.765 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.762+0000 7f66ecff9700 1 -- 172.21.15.42:0/3661036248 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180949 (secure 0 0 0) 0x7f66e00966e0 con 0x7f66f0100420 2023-12-12T10:09:36.903 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:36 smithi042 ceph-mon[98894]: pgmap v341: 129 pgs: 129 active+clean; 717 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 16 op/s 2023-12-12T10:09:36.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:36 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3012327189' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-12T10:09:36.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:36 smithi042 ceph-mon[98894]: osdmap e102: 12 total, 12 up, 12 in 2023-12-12T10:09:36.904 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:36 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/395962287' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-12T10:09:36.966 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:36.963+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66f0101b60 con 0x7f66f0100420 2023-12-12T10:09:36.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:36 smithi111 ceph-mon[102565]: pgmap v341: 129 pgs: 129 active+clean; 717 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 16 op/s 2023-12-12T10:09:36.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:36 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3012327189' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-12T10:09:36.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:36 smithi111 ceph-mon[102565]: osdmap e102: 12 total, 12 up, 12 in 2023-12-12T10:09:36.999 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:36 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/395962287' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-12T10:09:37.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:36 smithi173 ceph-mon[103308]: pgmap v341: 129 pgs: 129 active+clean; 717 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 1.9 KiB/s wr, 16 op/s 2023-12-12T10:09:37.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:36 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3012327189' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-12T10:09:37.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:36 smithi173 ceph-mon[103308]: osdmap e102: 12 total, 12 up, 12 in 2023-12-12T10:09:37.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:36 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/395962287' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-12T10:09:37.591 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.589+0000 7f66ecff9700 1 -- 172.21.15.42:0/3661036248 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v104) v1 ==== 178+0+0 (secure 0 0 0) 0x7f66e0099d00 con 0x7f66f0100420 2023-12-12T10:09:37.600 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.595+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f66dc0a15f0 msgr2=0x7f66dc0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:37.601 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.595+0000 7f66f61bc700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f66dc0a15f0 0x7f66dc0a3ab0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f66d800b2e0 tx=0x7f66d8005fd0 comp rx=0 tx=0).stop 2023-12-12T10:09:37.601 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.595+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 msgr2=0x7f66f019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-12T10:09:37.601 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.595+0000 7f66f61bc700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 0x7f66f019d6c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f66e000d980 tx=0x7f66e000dc90 comp rx=0 tx=0).stop 2023-12-12T10:09:37.601 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.598+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 shutdown_connections 2023-12-12T10:09:37.601 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.598+0000 7f66f61bc700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:6800/1776845983,v1:172.21.15.42:6801/1776845983] conn(0x7f66dc0a15f0 0x7f66dc0a3ab0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:37.601 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.598+0000 7f66f61bc700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f66f01064c0 0x7f66f01977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:37.601 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.598+0000 7f66f61bc700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f66f0100420 0x7f66f019d6c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:37.602 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.598+0000 7f66f61bc700 1 --2- 172.21.15.42:0/3661036248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f0100de0 0x7f66f019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-12T10:09:37.602 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.598+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 >> 172.21.15.42:0/3661036248 conn(0x7f66f00775a0 msgr2=0x7f66f0069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-12T10:09:37.602 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.599+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 shutdown_connections 2023-12-12T10:09:37.603 INFO:teuthology.orchestra.run.smithi042.stderr:2023-12-12T10:09:37.600+0000 7f66f61bc700 1 -- 172.21.15.42:0/3661036248 wait complete. 2023-12-12T10:09:37.607 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data_ec' removed 2023-12-12T10:09:37.625 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-12T10:09:37.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-12T10:09:37.740 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-12T10:09:37.793 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-12T10:09:37.793 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-12T10:09:37.811 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-12T10:09:37.828 DEBUG:teuthology.orchestra.run.smithi173:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-12T10:09:37.844 INFO:tasks.cephadm:Stopping all daemons... 2023-12-12T10:09:37.845 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-12-12T10:09:37.845 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.a 2023-12-12T10:09:37.874 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:37 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/395962287' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-12T10:09:37.874 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:37 smithi042 ceph-mon[98894]: osdmap e103: 12 total, 12 up, 12 in 2023-12-12T10:09:37.874 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:37 smithi042 ceph-mon[98894]: from='client.? 172.21.15.42:0/3661036248' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-12T10:09:37.997 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:37 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/395962287' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-12T10:09:37.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:37 smithi111 ceph-mon[102565]: osdmap e103: 12 total, 12 up, 12 in 2023-12-12T10:09:37.998 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:37 smithi111 ceph-mon[102565]: from='client.? 172.21.15.42:0/3661036248' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-12T10:09:38.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:37 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/395962287' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-12T10:09:38.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:37 smithi173 ceph-mon[103308]: osdmap e103: 12 total, 12 up, 12 in 2023-12-12T10:09:38.074 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:37 smithi173 ceph-mon[103308]: from='client.? 172.21.15.42:0/3661036248' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-12T10:09:38.154 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:37 smithi042 systemd[1]: Stopping Ceph mon.a for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:09:38.561 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:38 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-a[98870]: 2023-12-12T10:09:38.267+0000 7fc756442700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:09:38.561 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:38 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-a[98870]: 2023-12-12T10:09:38.267+0000 7fc756442700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-12-12T10:09:38.906 INFO:journalctl@ceph.mon.a.smithi042.stdout:Dec 12 10:09:38 smithi042 bash[276568]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-a 2023-12-12T10:09:39.017 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.a.service' 2023-12-12T10:09:39.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:09:39.110 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-12-12T10:09:39.110 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-12-12T10:09:39.110 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.b 2023-12-12T10:09:39.479 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:39 smithi111 systemd[1]: Stopping Ceph mon.b for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:09:39.480 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-b[102541]: 2023-12-12T10:09:39.474+0000 7f1cae8ed700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:09:39.480 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-b[102541]: 2023-12-12T10:09:39.474+0000 7f1cae8ed700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-12-12T10:09:40.135 INFO:journalctl@ceph.mon.b.smithi111.stdout:Dec 12 10:09:39 smithi111 bash[145748]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-b 2023-12-12T10:09:40.141 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.b.service' 2023-12-12T10:09:40.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:09:40.248 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-12-12T10:09:40.248 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-12-12T10:09:40.249 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.c 2023-12-12T10:09:40.573 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:40 smithi173 systemd[1]: Stopping Ceph mon.c for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:09:41.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-c[103284]: 2023-12-12T10:09:40.637+0000 7fbba15dc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:09:41.073 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-c[103284]: 2023-12-12T10:09:40.637+0000 7fbba15dc700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-12-12T10:09:41.558 INFO:journalctl@ceph.mon.c.smithi173.stdout:Dec 12 10:09:41 smithi173 bash[146359]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mon-c 2023-12-12T10:09:41.565 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mon.c.service' 2023-12-12T10:09:41.616 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:09:41.616 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-12-12T10:09:41.616 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-12-12T10:09:41.617 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.x 2023-12-12T10:09:41.903 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 10:09:41 smithi042 systemd[1]: Stopping Ceph mgr.x for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:09:42.217 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 10:09:41 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T10:09:41.958+0000 7f948c63a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:09:42.217 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 10:09:41 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x[99135]: 2023-12-12T10:09:41.958+0000 7f948c63a700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-12T10:09:42.500 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Dec 12 10:09:42 smithi042 bash[276708]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-x 2023-12-12T10:09:42.507 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.x.service' 2023-12-12T10:09:42.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:09:42.546 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-12-12T10:09:42.546 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-12-12T10:09:42.546 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.y 2023-12-12T10:09:42.975 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 10:09:42 smithi111 systemd[1]: Stopping Ceph mgr.y for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:09:43.246 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 10:09:43 smithi111 bash[145884]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-y 2023-12-12T10:09:43.425 INFO:journalctl@ceph.mgr.y.smithi111.stdout:Dec 12 10:09:43 smithi111 systemd[1]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-12-12T10:09:43.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.y.service' 2023-12-12T10:09:43.466 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:09:43.466 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-12-12T10:09:43.466 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-12-12T10:09:43.467 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.z 2023-12-12T10:09:43.823 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 10:09:43 smithi173 systemd[1]: Stopping Ceph mgr.z for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:09:44.173 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 10:09:44 smithi173 bash[146506]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mgr-z 2023-12-12T10:09:44.356 INFO:journalctl@ceph.mgr.z.smithi173.stdout:Dec 12 10:09:44 smithi173 systemd[1]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-12-12T10:09:44.359 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mgr.z.service' 2023-12-12T10:09:44.396 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:09:44.396 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-12-12T10:09:44.396 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-12-12T10:09:44.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.0 2023-12-12T10:09:44.760 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 10:09:44 smithi042 systemd[1]: Stopping Ceph osd.0 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:09:45.154 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 10:09:44 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-0[118191]: 2023-12-12T10:09:44.757+0000 7f6f31537700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:09:45.154 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 10:09:44 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-0[118191]: 2023-12-12T10:09:44.757+0000 7f6f31537700 -1 osd.0 104 *** Got signal Terminated *** 2023-12-12T10:09:45.154 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 10:09:44 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-0[118191]: 2023-12-12T10:09:44.757+0000 7f6f31537700 -1 osd.0 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:09:50.904 INFO:journalctl@ceph.osd.0.smithi042.stdout:Dec 12 10:09:50 smithi042 bash[276855]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-0 2023-12-12T10:09:53.362 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.0.service' 2023-12-12T10:09:53.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:09:53.418 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-12-12T10:09:53.418 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-12-12T10:09:53.418 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.1 2023-12-12T10:09:53.748 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 10:09:53 smithi111 systemd[1]: Stopping Ceph osd.1 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:09:54.248 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 10:09:53 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-1[107648]: 2023-12-12T10:09:53.790+0000 7f9b029a1700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:09:54.248 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 10:09:53 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-1[107648]: 2023-12-12T10:09:53.790+0000 7f9b029a1700 -1 osd.1 104 *** Got signal Terminated *** 2023-12-12T10:09:54.248 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 10:09:53 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-1[107648]: 2023-12-12T10:09:53.790+0000 7f9b029a1700 -1 osd.1 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:09:59.748 INFO:journalctl@ceph.osd.1.smithi111.stdout:Dec 12 10:09:59 smithi111 bash[146031]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-1 2023-12-12T10:10:01.971 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.1.service' 2023-12-12T10:10:02.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:10:02.012 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-12-12T10:10:02.013 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-12-12T10:10:02.013 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.2 2023-12-12T10:10:02.323 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 10:10:02 smithi173 systemd[1]: Stopping Ceph osd.2 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:10:02.823 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 10:10:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-2[109160]: 2023-12-12T10:10:02.377+0000 7fc88461a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:10:02.824 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 10:10:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-2[109160]: 2023-12-12T10:10:02.377+0000 7fc88461a700 -1 osd.2 104 *** Got signal Terminated *** 2023-12-12T10:10:02.824 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 10:10:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-2[109160]: 2023-12-12T10:10:02.377+0000 7fc88461a700 -1 osd.2 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:10:08.573 INFO:journalctl@ceph.osd.2.smithi173.stdout:Dec 12 10:10:08 smithi173 bash[146658]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-2 2023-12-12T10:10:09.819 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.2.service' 2023-12-12T10:10:09.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:10:09.860 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-12-12T10:10:09.861 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-12-12T10:10:09.861 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.3 2023-12-12T10:10:10.153 INFO:journalctl@ceph.osd.3.smithi042.stdout:Dec 12 10:10:09 smithi042 systemd[1]: Stopping Ceph osd.3 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:10:10.654 INFO:journalctl@ceph.osd.3.smithi042.stdout:Dec 12 10:10:10 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-3[126302]: 2023-12-12T10:10:10.212+0000 7f8cfe8cb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:10:10.654 INFO:journalctl@ceph.osd.3.smithi042.stdout:Dec 12 10:10:10 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-3[126302]: 2023-12-12T10:10:10.212+0000 7f8cfe8cb700 -1 osd.3 104 *** Got signal Terminated *** 2023-12-12T10:10:10.654 INFO:journalctl@ceph.osd.3.smithi042.stdout:Dec 12 10:10:10 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-3[126302]: 2023-12-12T10:10:10.212+0000 7f8cfe8cb700 -1 osd.3 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:10:10.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:10 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:10.551+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:11.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:11.552+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:12.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:12 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:12.390+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:12.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:12 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:12.519+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:13.153 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:12 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:12.707+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:13.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:13 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:13.384+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:13.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:13 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:13.470+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:14.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:13 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:13.741+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:14.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:13 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:13.867+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:14.747 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:14 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:14.340+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:14.762 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:14.507+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:15.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:14.759+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:15.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:14.838+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:15.102 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:14 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:14.783+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:15.404 INFO:journalctl@ceph.osd.3.smithi042.stdout:Dec 12 10:10:15 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-3[126302]: 2023-12-12T10:10:15.099+0000 7f8cfaec6700 -1 osd.3 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:49.709697+0000 front 2023-12-12T10:09:49.709741+0000 (oldest deadline 2023-12-12T10:10:14.310441+0000) 2023-12-12T10:10:15.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:15 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:15.377+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:15.759 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:15.497+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:16.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:15.876+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:16.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:15.757+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:16.154 INFO:journalctl@ceph.osd.3.smithi042.stdout:Dec 12 10:10:15 smithi042 bash[277144]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-3 2023-12-12T10:10:16.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:15 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:15.775+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:16.737 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:16 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:16.456+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:16.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:16 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:16.364+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:17.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:16 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:16.910+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:17.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:16 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:16.734+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:17.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:16 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:16.806+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:17.492 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.3.service' 2023-12-12T10:10:17.533 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:10:17.533 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-12-12T10:10:17.533 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-12-12T10:10:17.534 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.4 2023-12-12T10:10:17.748 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 10:10:17 smithi111 systemd[1]: Stopping Ceph osd.4 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:10:17.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:17 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:17.338+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:17.824 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:17 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:17.743+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:17.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:17 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:17.499+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:18.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:17 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:17.771+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:18.248 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 10:10:17 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4[116403]: 2023-12-12T10:10:17.884+0000 7f728e440700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:10:18.248 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 10:10:17 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4[116403]: 2023-12-12T10:10:17.884+0000 7f728e440700 -1 osd.4 104 *** Got signal Terminated *** 2023-12-12T10:10:18.248 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 10:10:17 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4[116403]: 2023-12-12T10:10:17.884+0000 7f728e440700 -1 osd.4 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:10:18.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:17 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:17.881+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:18.747 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:18 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:18.346+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:18.823 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:18 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:18.698+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:18.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:18 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:18.542+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:19.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:18 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:18.741+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:19.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:18 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:18.904+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:19.747 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:19 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:19.317+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:19.823 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:19 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:19.732+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:19.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:19 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:19.513+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:20.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:19 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:19.770+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:20.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:19 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:19.888+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:20.498 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 10:10:20 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4[116403]: 2023-12-12T10:10:20.233+0000 7f728aa3b700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.012387+0000 front 2023-12-12T10:09:58.012357+0000 (oldest deadline 2023-12-12T10:10:19.712517+0000) 2023-12-12T10:10:20.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:20 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:20.319+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:20.739 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:20 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:20.470+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:20.739 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:20 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:20.470+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:21.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:20 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:20.926+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:21.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:20 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:20.926+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:21.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:20 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:20.737+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:21.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:20 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:20.737+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:21.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:20 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:20.746+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:21.451 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 10:10:21 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4[116403]: 2023-12-12T10:10:21.193+0000 7f728aa3b700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.012387+0000 front 2023-12-12T10:09:58.012357+0000 (oldest deadline 2023-12-12T10:10:19.712517+0000) 2023-12-12T10:10:21.451 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:21 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:21.351+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:21.747 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:21 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:21.448+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:21.785 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:21 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:21.431+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:21.785 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:21 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:21.431+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:22.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:21 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:21.783+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:22.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:21 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:21.783+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:22.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:21 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:21.914+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:22.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:21 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:21.914+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:22.153 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:21 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:21.784+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:22.498 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 10:10:22 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4[116403]: 2023-12-12T10:10:22.169+0000 7f728aa3b700 -1 osd.4 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.012387+0000 front 2023-12-12T10:09:58.012357+0000 (oldest deadline 2023-12-12T10:10:19.712517+0000) 2023-12-12T10:10:22.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:22 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:22.357+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:22.498 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:22 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:22.405+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:22.764 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:22 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:22.427+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:22.764 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:22 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:22.427+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:23.011 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:22 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:22.734+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:23.011 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:22 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:22.734+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:23.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:22 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:22.761+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:23.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:22 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:22.761+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:23.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:22 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:22.957+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:23.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:22 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:22.957+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:23.404 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:23 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:23.008+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:23.594 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:23 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:23.422+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:23.594 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:23 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:23.321+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:23.594 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:23 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:23.321+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:23.771 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:23 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:23.464+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:23.771 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:23 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:23.464+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:23.998 INFO:journalctl@ceph.osd.4.smithi111.stdout:Dec 12 10:10:23 smithi111 bash[146315]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-4 2023-12-12T10:10:24.053 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:23 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:23.725+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:24.053 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:23 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:23.725+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:24.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:24 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:24.006+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:24.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:24 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:24.006+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:24.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:23 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:23.768+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:24.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:23 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:23.768+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:24.403 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:24 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:24.050+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:24.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:24 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:24.402+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:24.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:24 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:24.283+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:24.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:24 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:24.283+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:24.774 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:24 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:24.513+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:24.775 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:24 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:24.513+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:25.045 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:24 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:24.743+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:25.046 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:24 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:24.743+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:25.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.4.service' 2023-12-12T10:10:25.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:24 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:24.772+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:25.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:24 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:24.772+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:25.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:25.021+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:25.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:25.021+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:25.101 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:10:25.102 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-12-12T10:10:25.102 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-12-12T10:10:25.102 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.5 2023-12-12T10:10:25.371 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:25 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:25.270+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:25.371 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:25 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:25.270+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:25.404 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:25 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:25.043+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:25.458 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:25 smithi173 systemd[1]: Stopping Ceph osd.5 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:10:25.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:25 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:25.368+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:25.790 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:25.455+0000 7f4fbd916700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:10:25.790 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:25.455+0000 7f4fbd916700 -1 osd.5 104 *** Got signal Terminated *** 2023-12-12T10:10:25.790 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:25.455+0000 7f4fbd916700 -1 osd.5 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:10:25.790 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:25.510+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:25.791 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:25.510+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:25.999 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:25 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:25.694+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:26.000 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:25 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:25.694+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:26.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:25.787+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:26.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:25.787+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:26.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:25.971+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:26.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:25 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:25.971+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:26.404 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:25 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:25.996+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:26.498 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:26 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:26.375+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:26.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:26 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:26.226+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:26.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:26 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:26.226+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:26.770 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:26 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:26.471+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:26.771 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:26 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:26.471+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:27.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:26 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:26.768+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:27.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:26 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:26.768+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:27.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:26 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:26.942+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:27.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:26 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:26.942+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:27.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:26 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:26.957+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:27.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:26 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:26.711+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:27.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:26 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:26.711+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:27.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:27 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:27.377+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:27.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:27 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:27.246+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:27.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:27 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:27.246+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:27.759 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:27 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:27.499+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:27.759 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:27 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:27.499+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:28.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:27 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:27.756+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:28.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:27 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:27.756+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:28.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:27 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:27.939+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:28.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:27 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:27.939+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:28.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:27 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:27.965+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:28.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:27 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:27.745+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:28.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:27 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:27.745+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:28.725 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:28.453+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:28.725 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:28.453+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:28.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:28 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:28.336+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:28.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:28 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:28.265+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:28.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:28 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:28.265+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:29.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:28.722+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:29.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:28.722+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:29.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:28.909+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:29.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:28 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:28.909+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:29.153 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:28 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:28.942+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:29.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:28 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:28.721+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:29.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:28 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:28.721+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:29.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:29.337+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:29.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:29.310+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:29.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:29.310+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:29.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:29 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:29.310+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:29.755 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:29.468+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:29.756 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:29.468+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:30.073 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:29.753+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:48.378896+0000 front 2023-12-12T10:09:48.378823+0000 (oldest deadline 2023-12-12T10:10:14.278817+0000) 2023-12-12T10:10:30.074 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5[117107]: 2023-12-12T10:10:29.753+0000 7f4fb770c700 -1 osd.5 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.980207+0000 front 2023-12-12T10:09:55.980171+0000 (oldest deadline 2023-12-12T10:10:20.680140+0000) 2023-12-12T10:10:30.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:29.933+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:30.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:29 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:29.933+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:30.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:29 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:29.950+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:30.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:29 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:29.950+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:03.361142+0000 front 2023-12-12T10:10:03.360859+0000 (oldest deadline 2023-12-12T10:10:29.260933+0000) 2023-12-12T10:10:30.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:29 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:29.760+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:30.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:29 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:29.760+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:30.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:30 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:30.376+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:30.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:30 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:30.342+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:30.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:30 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:30.342+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:30.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:30 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:30.342+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:30.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:30 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:30.514+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:30.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:30 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:30.514+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:31.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:30.910+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:31.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:30.910+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:03.361142+0000 front 2023-12-12T10:10:03.360859+0000 (oldest deadline 2023-12-12T10:10:29.260933+0000) 2023-12-12T10:10:31.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:30.744+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:31.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:30.744+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:31.174 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:30 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:30.902+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:31.175 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:30 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:30.902+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:31.472 INFO:journalctl@ceph.osd.5.smithi173.stdout:Dec 12 10:10:31 smithi173 bash[146933]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-5 2023-12-12T10:10:31.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:31 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:31.345+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:31.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:31 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:31.345+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:31.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:31 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:31.302+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:31.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:31 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:31.302+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:31.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:31 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:31.302+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:31.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:31 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:31.469+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:31.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:31 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:31.469+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:31.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:31 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:31.469+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:32.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:31.876+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:32.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:31.876+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:03.361142+0000 front 2023-12-12T10:10:03.360859+0000 (oldest deadline 2023-12-12T10:10:29.260933+0000) 2023-12-12T10:10:32.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:31.742+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:32.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:31.742+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:32.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:31 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:31.742+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:32.306 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:31 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:31.908+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:32.307 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:31 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:31.908+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:32.307 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:31 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:31.908+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:32.573 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:32.513+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:32.573 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:32.513+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:32.574 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:32.513+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:32.721 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.5.service' 2023-12-12T10:10:32.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:32.388+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:32.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:32.388+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:32.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:32.304+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:32.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:32.304+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:32.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:32 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:32.304+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:32.760 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:10:32.760 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-12-12T10:10:32.760 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-12-12T10:10:32.761 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.6 2023-12-12T10:10:33.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:32.927+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:33.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:32.927+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:33.074 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:32 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:32.927+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:33.121 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:32 smithi042 systemd[1]: Stopping Ceph osd.6 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:10:33.121 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:32 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:32.891+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:33.121 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:32 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:32.891+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:03.361142+0000 front 2023-12-12T10:10:03.360859+0000 (oldest deadline 2023-12-12T10:10:29.260933+0000) 2023-12-12T10:10:33.122 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:33.118+0000 7fae0a93d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:10:33.122 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:32 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:32.724+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:33.122 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:32 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:32.724+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:33.122 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:32 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:32.724+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:33.403 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:33.118+0000 7fae0a93d700 -1 osd.6 104 *** Got signal Terminated *** 2023-12-12T10:10:33.404 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:33.118+0000 7fae0a93d700 -1 osd.6 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:10:33.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:33.359+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:33.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:33.359+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:33.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:33.298+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:33.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:33.298+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:33.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:33.298+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:33.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:33.530+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:33.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:33.530+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:33.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:33.530+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:34.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:33.885+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:34.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:33.885+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:03.361142+0000 front 2023-12-12T10:10:03.360859+0000 (oldest deadline 2023-12-12T10:10:29.260933+0000) 2023-12-12T10:10:34.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:33.765+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:34.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:33.765+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:34.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:33 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:33.765+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:34.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:33.964+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:34.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:33.964+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:34.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:33 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:33.964+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:34.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:34 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:34.377+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:34.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:34 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:34.377+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:34.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:34 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:34.306+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:34.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:34 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:34.306+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:34.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:34 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:34.306+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:34.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:34 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:34.554+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:34.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:34 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:34.554+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:34.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:34 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:34.554+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:35.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:34.918+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:35.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:34.918+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:03.361142+0000 front 2023-12-12T10:10:03.360859+0000 (oldest deadline 2023-12-12T10:10:29.260933+0000) 2023-12-12T10:10:35.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:34.759+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:35.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:34.759+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:35.155 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:34 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:34.759+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:35.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:35.012+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:35.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:35.012+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:35.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:35.012+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:35.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:35 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:35.398+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:35.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:35 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:35.398+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:35.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:35 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:35.258+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:35.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:35 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:35.258+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:35.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:35 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:35.258+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:35.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:35.536+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:35.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:35.536+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:35.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:35 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:35.536+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:36.153 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:35.916+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:36.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:35.916+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:03.361142+0000 front 2023-12-12T10:10:03.360859+0000 (oldest deadline 2023-12-12T10:10:29.260933+0000) 2023-12-12T10:10:36.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:35.758+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:36.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:35.758+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:36.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:35 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:35.758+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:36.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:36 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:36.029+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:36.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:36 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:36.029+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:36.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:36 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:36.029+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:36.498 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:36 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:36.408+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:36.498 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:36 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:36.408+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:36.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:36 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:36.208+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:36.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:36 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:36.208+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:36.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:36 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:36.208+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:36.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:36 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:36.208+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:36.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:36 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:36.542+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:36.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:36 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:36.542+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:36.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:36 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:36.542+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:37.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:36.722+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:37.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:36.722+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:37.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:36.722+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:37.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:36.902+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:37.155 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:36.902+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:03.361142+0000 front 2023-12-12T10:10:03.360859+0000 (oldest deadline 2023-12-12T10:10:29.260933+0000) 2023-12-12T10:10:37.155 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:36 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:36.902+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:15.162145+0000 front 2023-12-12T10:10:15.162218+0000 (oldest deadline 2023-12-12T10:10:36.862248+0000) 2023-12-12T10:10:37.498 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:37.408+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:37.498 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:37.408+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:37.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:37.211+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:37.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:37.211+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:37.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:37.211+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:37.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:37 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:37.211+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:37.509 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:37.073+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:37.509 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:37.073+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:37.509 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:37.073+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:37.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:37.506+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:37.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:37.506+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:37.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:37 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:37.506+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:38.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:37.720+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:38.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:37.720+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:38.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:37.720+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:38.154 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:37.869+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.259476+0000 front 2023-12-12T10:09:58.259344+0000 (oldest deadline 2023-12-12T10:10:22.359307+0000) 2023-12-12T10:10:38.155 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:37.869+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:03.361142+0000 front 2023-12-12T10:10:03.360859+0000 (oldest deadline 2023-12-12T10:10:29.260933+0000) 2023-12-12T10:10:38.155 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:37 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6[134246]: 2023-12-12T10:10:37.869+0000 7fae04f34700 -1 osd.6 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:15.162145+0000 front 2023-12-12T10:10:15.162218+0000 (oldest deadline 2023-12-12T10:10:36.862248+0000) 2023-12-12T10:10:38.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:38.026+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:38.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:38.026+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:38.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:38.026+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:38.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:38.026+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:38.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:38.369+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:38.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:38.369+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:38.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:38.260+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:38.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:38.260+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:38.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:38.260+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:38.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:38.260+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:38.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:38.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:38.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:38.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:38.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:38 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:38.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:38.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:38 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:38.766+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:38.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:38 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:38.766+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:38.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:38 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:38.766+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:38.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:38 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:38.766+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:39.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:39.059+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:39.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:39.059+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:39.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:39.059+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:39.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:39.059+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:39.404 INFO:journalctl@ceph.osd.6.smithi042.stdout:Dec 12 10:10:39 smithi042 bash[277420]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-6 2023-12-12T10:10:39.498 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:39.352+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:39.498 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:39.352+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:39.498 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:39.352+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:15.133792+0000 front 2023-12-12T10:10:15.133887+0000 (oldest deadline 2023-12-12T10:10:39.233807+0000) 2023-12-12T10:10:39.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:39.244+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:39.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:39.244+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:39.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:39.244+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:39.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:39 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:39.244+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:39.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:39.431+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:39.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:39.431+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:39.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:39 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:39.431+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:39.903 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:39 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:39.747+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:39.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:39 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:39.747+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:39.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:39 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:39.747+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:39.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:39 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:39.747+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:40.376 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.6.service' 2023-12-12T10:10:40.395 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:40.078+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:40.395 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:40.078+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:40.395 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:40.078+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:40.395 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:40.078+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:40.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:10:40.418 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-12-12T10:10:40.418 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-12-12T10:10:40.418 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.7 2023-12-12T10:10:40.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:40.225+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:40.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:40.225+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:40.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:40.225+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:40.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:40.225+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:40.499 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:40.393+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:40.499 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:40.393+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:40.499 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:40.393+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:15.133792+0000 front 2023-12-12T10:10:15.133887+0000 (oldest deadline 2023-12-12T10:10:39.233807+0000) 2023-12-12T10:10:40.499 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:40 smithi111 systemd[1]: Stopping Ceph osd.7 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:10:40.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:40.393+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:40.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:40.393+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:40.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:40 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:40.393+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:41.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:40 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:40.702+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:41.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:40 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:40.702+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:41.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:40 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:40.702+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:41.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:40 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:40.702+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:41.248 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:40.801+0000 7f78ec4fb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:10:41.248 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:40.801+0000 7f78ec4fb700 -1 osd.7 104 *** Got signal Terminated *** 2023-12-12T10:10:41.248 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:40 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:40.801+0000 7f78ec4fb700 -1 osd.7 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:10:41.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:41 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:41.041+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:41.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:41 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:41.041+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:41.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:41 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:41.041+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:41.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:41 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:41.041+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:41.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:41 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:41.423+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:41.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:41 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:41.423+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:41.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:41 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:41.423+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:15.133792+0000 front 2023-12-12T10:10:15.133887+0000 (oldest deadline 2023-12-12T10:10:39.233807+0000) 2023-12-12T10:10:41.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:41 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:41.267+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:41.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:41 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:41.267+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:41.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:41 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:41.267+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:41.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:41 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:41.267+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:41.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:41 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:41.408+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:41.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:41 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:41.408+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:41.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:41 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:41.408+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:42.153 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:41 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:41.664+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:42.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:41 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:41.664+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:42.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:41 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:41.664+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:42.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:41 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:41.664+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:42.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:42 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:42.055+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:42.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:42 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:42.055+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:42.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:42 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:42.055+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:42.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:42 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:42.055+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:42.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:42.413+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:42.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:42.413+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:42.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:42.413+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:15.133792+0000 front 2023-12-12T10:10:15.133887+0000 (oldest deadline 2023-12-12T10:10:39.233807+0000) 2023-12-12T10:10:42.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:42.260+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:42.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:42.260+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:42.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:42.260+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:42.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:42.260+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:42.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:42 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:42.431+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:42.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:42 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:42.431+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:42.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:42 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:42.431+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:42.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:42 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:42.619+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:42.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:42 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:42.619+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:42.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:42 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:42.619+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:42.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:42 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:42.619+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:43.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:43 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:43.063+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:43.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:43 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:43.063+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:43.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:43 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:43.063+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:43.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:43 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:43.063+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:43.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:43 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:43.418+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:43.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:43 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:43.418+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:43.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:43 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:43.418+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:15.133792+0000 front 2023-12-12T10:10:15.133887+0000 (oldest deadline 2023-12-12T10:10:39.233807+0000) 2023-12-12T10:10:43.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:43 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:43.252+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:43.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:43 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:43.252+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:43.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:43 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:43.252+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:43.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:43 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:43.252+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:43.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:43 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:43.424+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:43.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:43 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:43.424+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:43.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:43 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:43.424+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:43.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:43 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:43.630+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:43.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:43 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:43.630+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:43.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:43 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:43.630+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:43.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:43 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:43.630+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:44.393 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:44 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:44.093+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:44.393 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:44 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:44.093+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:44.393 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:44 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:44.093+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:44.393 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:44 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:44.093+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:44.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:44 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:44.421+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:44.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:44 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:44.421+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:44.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:44 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:44.421+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:15.133792+0000 front 2023-12-12T10:10:15.133887+0000 (oldest deadline 2023-12-12T10:10:39.233807+0000) 2023-12-12T10:10:44.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:44 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:44.268+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:44.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:44 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:44.268+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:44.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:44 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:44.268+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:44.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:44 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:44.268+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:44.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:44 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:44.390+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:44.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:44 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:44.390+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:44.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:44 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:44.390+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:44.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:44 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:44.640+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:44.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:44 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:44.640+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:44.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:44 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:44.640+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:44.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:44 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:44.640+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:44.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:44 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:44.640+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:45.413 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:45 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:45.092+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:45.414 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:45 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:45.092+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:45.414 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:45 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:45.092+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:45.414 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:45 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:45.092+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:45.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:45 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:45.449+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.830254+0000 front 2023-12-12T10:09:55.830319+0000 (oldest deadline 2023-12-12T10:10:21.130461+0000) 2023-12-12T10:10:45.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:45 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:45.449+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.031717+0000 front 2023-12-12T10:10:07.031684+0000 (oldest deadline 2023-12-12T10:10:30.531703+0000) 2023-12-12T10:10:45.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:45 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:45.449+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:15.133792+0000 front 2023-12-12T10:10:15.133887+0000 (oldest deadline 2023-12-12T10:10:39.233807+0000) 2023-12-12T10:10:45.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:45 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7[125154]: 2023-12-12T10:10:45.449+0000 7f78e8af6700 -1 osd.7 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.234354+0000 front 2023-12-12T10:10:19.234331+0000 (oldest deadline 2023-12-12T10:10:44.534388+0000) 2023-12-12T10:10:45.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:45 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:45.316+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:45.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:45 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:45.316+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:45.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:45 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:45.316+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:45.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:45 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:45.316+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:45.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:45 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:45.411+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:45.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:45 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:45.411+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:45.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:45 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:45.411+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:45.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:45 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:45.620+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:45.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:45 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:45.620+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:45.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:45 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:45.620+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:45.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:45 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:45.620+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:45.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:45 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:45.620+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:46.464 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:46 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:46.135+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:46.464 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:46 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:46.136+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:46.464 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:46 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:46.136+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:46.464 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:46 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:46.136+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:46.748 INFO:journalctl@ceph.osd.7.smithi111.stdout:Dec 12 10:10:46 smithi111 bash[146594]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-7 2023-12-12T10:10:46.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:46 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:46.354+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:46.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:46 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:46.354+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:46.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:46 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:46.354+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:46.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:46 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:46.354+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:46.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:46 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:46.354+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:46.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:46 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:46.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:46.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:46 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:46.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:46.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:46 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:46.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:46.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:46 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:46.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:46.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:46 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:46.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:46.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:46 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:46.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:46.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:46 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:46.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:46.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:46 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:46.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:47.429 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:47 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:47.105+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:47.429 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:47 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:47.105+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:47.429 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:47 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:47.105+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:47.429 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:47 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:47.105+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:47.734 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:47 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:47.366+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:47.734 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:47 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:47.366+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:47.734 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:47 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:47.366+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:47.734 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:47 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:47.366+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:47.735 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:47 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:47.366+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:47.741 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.7.service' 2023-12-12T10:10:47.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:10:47.781 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-12-12T10:10:47.781 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-12-12T10:10:47.781 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.8 2023-12-12T10:10:47.811 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:47 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:47.426+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:47.811 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:47 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:47.426+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:47.812 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:47 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:47.426+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:47.812 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:47 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:47.426+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:47.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:47 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:47.631+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:47.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:47 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:47.631+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:47.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:47 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:47.631+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:47.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:47 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:47.631+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:47.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:47 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:47.631+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:48.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:47 smithi173 systemd[1]: Stopping Ceph osd.8 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:10:48.441 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:48.135+0000 7f1da2834700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:10:48.441 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:48.135+0000 7f1da2834700 -1 osd.8 104 *** Got signal Terminated *** 2023-12-12T10:10:48.441 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:48.135+0000 7f1da2834700 -1 osd.8 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:10:48.442 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:48.144+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:48.442 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:48.144+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:48.442 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:48.144+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:48.442 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:48.144+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:48.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:48 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:48.339+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:48.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:48 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:48.339+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:48.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:48 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:48.339+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:48.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:48 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:48.339+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:48.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:48 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:48.339+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:48.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:48.438+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:48.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:48.438+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:48.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:48.438+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:48.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:48 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:48.438+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:48.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:48 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:48.621+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:48.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:48 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:48.621+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:48.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:48 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:48.621+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:48.905 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:48 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:48.621+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:48.905 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:48 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:48.621+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:49.409 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:49 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:49.102+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:49.409 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:49 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:49.102+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:49.409 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:49 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:49.102+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:49.409 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:49 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:49.102+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:49.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:49 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:49.332+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:49.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:49 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:49.332+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:49.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:49 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:49.332+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:49.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:49 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:49.332+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:49.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:49 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:49.332+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:49.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:49 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:49.406+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:49.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:49 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:49.406+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:49.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:49 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:49.406+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:49.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:49 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:49.406+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:49.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:49 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:49.606+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:49.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:49 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:49.606+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:49.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:49 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:49.606+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:49.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:49 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:49.606+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:49.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:49 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:49.606+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:50.323 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:50.058+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:50.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:50.058+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:50.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:50.058+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:50.324 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:50.058+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:50.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:50 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:50.322+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:50.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:50 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:50.322+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:50.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:50 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:50.322+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:50.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:50 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:50.322+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:50.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:50 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:50.322+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:50.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:50.358+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:50.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:50.358+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:50.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:50.358+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:50.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:50.358+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:50.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:50.571+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:50.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:50.571+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:50.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:50.571+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:50.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:50.571+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:50.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:50 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:50.571+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:51.388 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:51 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:51.106+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:51.388 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:51 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:51.106+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:51.388 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:51 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:51.106+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:51.388 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:51 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:51.106+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:51.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:51 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:51.321+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:51.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:51 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:51.321+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:51.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:51 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:51.321+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:51.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:51 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:51.321+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:51.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:51 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:51.321+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:51.749 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:51 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:51.321+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:10:51.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:51 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:51.385+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:51.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:51 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:51.385+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:51.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:51 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:51.385+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:51.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:51 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:51.385+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:51.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:51 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:51.583+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:51.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:51 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:51.583+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:51.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:51 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:51.583+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:51.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:51 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:51.583+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:51.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:51 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:51.583+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:52.379 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:52 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:52.121+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.075847+0000 front 2023-12-12T10:09:47.075945+0000 (oldest deadline 2023-12-12T10:10:12.975850+0000) 2023-12-12T10:10:52.379 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:52 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:52.121+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:55.277155+0000 front 2023-12-12T10:09:55.276999+0000 (oldest deadline 2023-12-12T10:10:19.976902+0000) 2023-12-12T10:10:52.379 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:52 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:52.121+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:04.077827+0000 front 2023-12-12T10:10:04.077930+0000 (oldest deadline 2023-12-12T10:10:31.678617+0000) 2023-12-12T10:10:52.379 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:52 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8[125257]: 2023-12-12T10:10:52.121+0000 7f1d9d62c700 -1 osd.8 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.979859+0000 front 2023-12-12T10:10:13.979762+0000 (oldest deadline 2023-12-12T10:10:37.479868+0000) 2023-12-12T10:10:52.747 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:52 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:52.278+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:52.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:52 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:52.278+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:52.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:52 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:52.278+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:52.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:52 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:52.278+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:52.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:52 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:52.278+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:52.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:52 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:52.278+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:10:52.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:52 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:52.376+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:52.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:52 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:52.376+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:52.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:52 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:52.376+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:52.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:52 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:52.376+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:52.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:52 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:52.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:52.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:52 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:52.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:52.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:52 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:52.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:52.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:52 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:52.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:52.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:52 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:52.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:52.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:52 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:52.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:29.558590+0000 front 2023-12-12T10:10:29.558654+0000 (oldest deadline 2023-12-12T10:10:51.858506+0000) 2023-12-12T10:10:53.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:53 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:53.231+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:53.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:53 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:53.231+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:53.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:53 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:53.231+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:53.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:53 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:53.231+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:53.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:53 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:53.231+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:53.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:53 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:53.231+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:10:53.814 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:53 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:53.393+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:53.814 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:53 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:53.393+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:53.814 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:53 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:53.393+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:53.815 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:53 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:53.393+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:53.815 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:53 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:53.393+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:10:53.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:53 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:53.535+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:53.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:53 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:53.535+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:53.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:53 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:53.535+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:53.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:53 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:53.535+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:53.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:53 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:53.535+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:53.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:53 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:53.535+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:29.558590+0000 front 2023-12-12T10:10:29.558654+0000 (oldest deadline 2023-12-12T10:10:51.858506+0000) 2023-12-12T10:10:54.073 INFO:journalctl@ceph.osd.8.smithi173.stdout:Dec 12 10:10:53 smithi173 bash[147202]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-8 2023-12-12T10:10:54.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:54 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:54.187+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:54.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:54 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:54.187+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:54.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:54 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:54.187+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:54.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:54 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:54.187+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:54.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:54 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:54.187+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:54.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:54 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:54.187+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:10:54.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:54 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:54.373+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:54.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:54 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:54.373+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:54.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:54 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:54.373+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:54.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:54 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:54.373+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:54.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:54 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:54.373+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:10:54.903 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:54.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:54.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:54.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:54.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:54.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:54.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:54.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:54.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:54.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:54.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:54 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:54.540+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:29.558590+0000 front 2023-12-12T10:10:29.558654+0000 (oldest deadline 2023-12-12T10:10:51.858506+0000) 2023-12-12T10:10:55.398 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.8.service' 2023-12-12T10:10:55.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:10:55.477 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-12-12T10:10:55.478 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-12-12T10:10:55.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.9 2023-12-12T10:10:55.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:55 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:55.186+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:55.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:55 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:55.186+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:55.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:55 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:55.186+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:55.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:55 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:55.186+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:55.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:55 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:55.186+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:55.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:55 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:55.186+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:10:55.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:55 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:55.406+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:55.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:55 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:55.406+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:55.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:55 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:55.406+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:55.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:55 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:55.406+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:55.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:55 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:55.406+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:10:55.852 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 systemd[1]: Stopping Ceph osd.9 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:10:55.852 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:55.527+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:55.853 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:55.527+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:55.853 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:55.527+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:55.853 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:55.527+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:55.853 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:55.527+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:55.853 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:55.527+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:29.558590+0000 front 2023-12-12T10:10:29.558654+0000 (oldest deadline 2023-12-12T10:10:51.858506+0000) 2023-12-12T10:10:56.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:55.849+0000 7fa922864700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:10:56.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:55.849+0000 7fa922864700 -1 osd.9 104 *** Got signal Terminated *** 2023-12-12T10:10:56.154 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:55 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:55.849+0000 7fa922864700 -1 osd.9 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:10:56.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:56 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:56.150+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:56.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:56 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:56.150+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:56.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:56 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:56.150+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:56.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:56 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:56.150+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:56.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:56 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:56.150+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:56.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:56 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:56.150+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:10:56.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:56 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:56.411+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:56.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:56 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:56.411+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:56.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:56 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:56.411+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:56.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:56 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:56.411+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:56.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:56 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:56.411+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:10:56.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:56.563+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:56.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:56.563+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:56.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:56.563+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:56.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:56.563+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:56.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:56.563+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:56.905 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:56 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:56.563+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:29.558590+0000 front 2023-12-12T10:10:29.558654+0000 (oldest deadline 2023-12-12T10:10:51.858506+0000) 2023-12-12T10:10:57.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:57 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:57.160+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:57.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:57 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:57.160+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:57.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:57 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:57.160+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:57.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:57 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:57.160+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:57.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:57 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:57.160+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:57.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:57 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:57.160+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:10:57.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:57 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:57.458+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:57.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:57 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:57.458+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:57.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:57 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:57.458+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:57.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:57 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:57.458+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:57.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:57 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:57.458+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:10:57.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:57 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:57.607+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:57.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:57 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:57.607+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:57.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:57 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:57.607+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:57.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:57 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:57.607+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:57.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:57 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:57.607+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:57.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:57 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:57.607+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:29.558590+0000 front 2023-12-12T10:10:29.558654+0000 (oldest deadline 2023-12-12T10:10:51.858506+0000) 2023-12-12T10:10:58.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:58 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:58.202+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:58.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:58 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:58.202+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:58.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:58 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:58.202+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:58.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:58 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:58.202+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:58.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:58 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:58.202+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:58.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:58 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:58.202+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:10:58.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:58 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:58.457+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:58.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:58 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:58.457+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:58.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:58 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:58.457+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:58.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:58 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:58.457+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:58.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:58 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:58.457+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:10:58.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:58 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:58.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:58.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:58 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:58.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:58.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:58 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:58.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:58.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:58 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:58.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:58.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:58 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:58.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:58.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:58 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:58.605+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:29.558590+0000 front 2023-12-12T10:10:29.558654+0000 (oldest deadline 2023-12-12T10:10:51.858506+0000) 2023-12-12T10:10:59.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:59 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:59.154+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:10:59.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:59 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:59.154+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:10:59.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:59 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:59.154+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:10:59.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:59 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:59.154+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:10:59.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:59 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:59.154+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:10:59.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:10:59 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:10:59.154+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:10:59.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:59 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:59.417+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:10:59.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:59 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:59.417+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:10:59.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:59 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:59.417+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:10:59.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:59 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:59.417+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:10:59.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:10:59 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:10:59.417+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:10:59.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:59 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:59.584+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:10:59.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:59 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:59.584+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:10:59.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:59 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:59.584+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:10:59.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:59 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:59.584+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:10:59.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:59 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:59.584+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:10:59.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:10:59 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:10:59.584+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:29.558590+0000 front 2023-12-12T10:10:29.558654+0000 (oldest deadline 2023-12-12T10:10:51.858506+0000) 2023-12-12T10:11:00.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:00 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:00.125+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:11:00.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:00 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:00.125+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:11:00.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:00 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:00.125+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:11:00.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:00 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:00.125+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:11:00.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:00 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:00.126+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:11:00.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:00 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:00.126+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:11:00.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:00 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:00.126+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:36.583242+0000 front 2023-12-12T10:10:36.583259+0000 (oldest deadline 2023-12-12T10:10:59.482949+0000) 2023-12-12T10:11:00.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:00 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:00.367+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:00.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:00 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:00.367+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:00.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:00 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:00.367+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:00.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:00 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:00.367+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:00.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:00 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:00.367+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:00.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:11:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:11:00.634+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.452064+0000 front 2023-12-12T10:09:47.451959+0000 (oldest deadline 2023-12-12T10:10:12.152136+0000) 2023-12-12T10:11:00.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:11:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:11:00.634+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.954018+0000 front 2023-12-12T10:09:57.953974+0000 (oldest deadline 2023-12-12T10:10:22.054022+0000) 2023-12-12T10:11:00.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:11:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:11:00.634+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:07.355069+0000 front 2023-12-12T10:10:07.355191+0000 (oldest deadline 2023-12-12T10:10:30.855150+0000) 2023-12-12T10:11:00.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:11:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:11:00.634+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:14.956151+0000 front 2023-12-12T10:10:14.956209+0000 (oldest deadline 2023-12-12T10:10:38.456235+0000) 2023-12-12T10:11:00.904 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:11:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:11:00.634+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:19.557407+0000 front 2023-12-12T10:10:19.557456+0000 (oldest deadline 2023-12-12T10:10:44.257366+0000) 2023-12-12T10:11:00.905 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:11:00 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9[143043]: 2023-12-12T10:11:00.634+0000 7fa91ee5f700 -1 osd.9 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:29.558590+0000 front 2023-12-12T10:10:29.558654+0000 (oldest deadline 2023-12-12T10:10:51.858506+0000) 2023-12-12T10:11:01.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:01 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:01.082+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:11:01.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:01 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:01.082+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:11:01.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:01 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:01.082+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:11:01.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:01 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:01.082+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:11:01.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:01 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:01.082+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:11:01.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:01 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:01.082+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:11:01.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:01 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:01.082+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:36.583242+0000 front 2023-12-12T10:10:36.583259+0000 (oldest deadline 2023-12-12T10:10:59.482949+0000) 2023-12-12T10:11:01.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:01 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:01.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:01.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:01 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:01.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:01.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:01 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:01.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:01.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:01 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:01.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:01.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:01 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:01.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:01.827 INFO:journalctl@ceph.osd.9.smithi042.stdout:Dec 12 10:11:01 smithi042 bash[277695]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-9 2023-12-12T10:11:02.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:02 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:02.079+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:11:02.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:02 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:02.079+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:11:02.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:02 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:02.079+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:11:02.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:02 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:02.079+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:11:02.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:02 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:02.079+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:11:02.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:02 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:02.079+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:11:02.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:02 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:02.079+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:36.583242+0000 front 2023-12-12T10:10:36.583259+0000 (oldest deadline 2023-12-12T10:10:59.482949+0000) 2023-12-12T10:11:02.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:02.447+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:02.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:02.447+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:02.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:02.447+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:02.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:02.447+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:02.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:02.447+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:02.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:02.447+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:03.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.9.service' 2023-12-12T10:11:03.141 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:03.142 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-12-12T10:11:03.142 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-12-12T10:11:03.142 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.10 2023-12-12T10:11:03.497 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.060+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:11:03.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.060+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:11:03.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.060+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:11:03.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.060+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:11:03.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.060+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:11:03.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.060+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:11:03.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.060+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:36.583242+0000 front 2023-12-12T10:10:36.583259+0000 (oldest deadline 2023-12-12T10:10:59.482949+0000) 2023-12-12T10:11:03.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 systemd[1]: Stopping Ceph osd.10 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:03.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.495+0000 7f756078b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:03.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.495+0000 7f756078b700 -1 osd.10 104 *** Got signal Terminated *** 2023-12-12T10:11:03.748 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:03 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:03.495+0000 7f756078b700 -1 osd.10 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:11:03.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:03 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:03.433+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:03.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:03 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:03.433+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:03.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:03 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:03.433+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:03.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:03 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:03.433+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:03.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:03 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:03.433+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:03.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:03 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:03.433+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:04.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:04 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:04.110+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:11:04.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:04 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:04.110+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:11:04.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:04 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:04.110+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:11:04.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:04 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:04.110+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:11:04.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:04 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:04.110+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:11:04.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:04 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:04.110+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:11:04.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:04 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:04.110+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:36.583242+0000 front 2023-12-12T10:10:36.583259+0000 (oldest deadline 2023-12-12T10:10:59.482949+0000) 2023-12-12T10:11:04.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:04 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:04.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:04.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:04 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:04.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:04.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:04 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:04.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:04.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:04 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:04.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:04.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:04 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:04.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:04.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:04 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:04.399+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:05.500 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:05 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:05.087+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:11:05.524 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:05 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:05.087+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:11:05.524 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:05 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:05.087+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:11:05.525 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:05 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:05.087+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:11:05.525 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:05 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:05.087+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:11:05.525 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:05 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:05.087+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:11:05.525 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:05 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:05.087+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:36.583242+0000 front 2023-12-12T10:10:36.583259+0000 (oldest deadline 2023-12-12T10:10:59.482949+0000) 2023-12-12T10:11:05.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:05 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:05.384+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:05.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:05 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:05.384+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:05.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:05 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:05.384+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:05.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:05 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:05.384+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:05.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:05 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:05.384+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:05.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:05 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:05.384+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:06.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:06 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:06.066+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:11:06.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:06 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:06.066+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:11:06.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:06 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:06.066+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:11:06.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:06 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:06.066+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:11:06.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:06 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:06.066+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:11:06.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:06 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:06.066+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:11:06.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:06 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:06.066+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:36.583242+0000 front 2023-12-12T10:10:36.583259+0000 (oldest deadline 2023-12-12T10:10:59.482949+0000) 2023-12-12T10:11:06.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:06 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:06.396+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:06.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:06 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:06.396+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:06.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:06 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:06.396+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:06.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:06 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:06.396+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:06.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:06 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:06.396+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:06.825 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:06 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:06.396+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:07.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:07 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:07.044+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:11:07.498 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:07 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:07.044+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:11:07.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:07 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:07.044+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:11:07.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:07 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:07.044+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:11:07.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:07 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:07.044+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:11:07.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:07 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:07.044+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:11:07.500 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:07 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:07.044+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:36.583242+0000 front 2023-12-12T10:10:36.583259+0000 (oldest deadline 2023-12-12T10:10:59.482949+0000) 2023-12-12T10:11:07.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:07 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:07.372+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:07.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:07 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:07.372+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:07.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:07 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:07.372+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:07.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:07 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:07.372+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:07.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:07 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:07.372+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:07.825 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:07 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:07.372+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:08.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:08.090+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:46.876063+0000 front 2023-12-12T10:09:46.876165+0000 (oldest deadline 2023-12-12T10:10:12.176000+0000) 2023-12-12T10:11:08.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:08.090+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:58.577790+0000 front 2023-12-12T10:09:58.577826+0000 (oldest deadline 2023-12-12T10:10:22.678017+0000) 2023-12-12T10:11:08.499 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:08.090+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:02.678582+0000 front 2023-12-12T10:10:02.678552+0000 (oldest deadline 2023-12-12T10:10:28.578589+0000) 2023-12-12T10:11:08.500 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:08.090+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2023-12-12T10:10:13.180010+0000 front 2023-12-12T10:10:13.179920+0000 (oldest deadline 2023-12-12T10:10:36.079959+0000) 2023-12-12T10:11:08.500 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:08.090+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:21.881196+0000 front 2023-12-12T10:10:21.881233+0000 (oldest deadline 2023-12-12T10:10:45.381368+0000) 2023-12-12T10:11:08.500 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:08.090+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:25.381948+0000 front 2023-12-12T10:10:25.381924+0000 (oldest deadline 2023-12-12T10:10:51.281779+0000) 2023-12-12T10:11:08.500 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:08 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10[133384]: 2023-12-12T10:11:08.090+0000 7f755cd86700 -1 osd.10 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:36.583242+0000 front 2023-12-12T10:10:36.583259+0000 (oldest deadline 2023-12-12T10:10:59.482949+0000) 2023-12-12T10:11:08.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:08 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:08.391+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:08.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:08 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:08.391+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:08.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:08 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:08.391+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:08.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:08 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:08.391+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:08.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:08 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:08.391+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:08.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:08 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:08.391+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:09.497 INFO:journalctl@ceph.osd.10.smithi111.stdout:Dec 12 10:11:09 smithi111 bash[146860]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-10 2023-12-12T10:11:09.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:09 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:09.360+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:09.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:09 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:09.360+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:09.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:09 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:09.360+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:09.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:09 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:09.360+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:09.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:09 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:09.360+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:09.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:09 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:09.360+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:10.686 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.10.service' 2023-12-12T10:11:10.726 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:10.726 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-12-12T10:11:10.726 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-12-12T10:11:10.726 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.11 2023-12-12T10:11:10.758 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:10 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:10.407+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:10.758 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:10 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:10.407+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:10.759 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:10 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:10.407+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:10.759 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:10 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:10.407+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:10.759 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:10 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:10.407+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:10.759 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:10 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:10.407+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:10.759 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:10 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:10.407+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2023-12-12T10:10:43.847256+0000 front 2023-12-12T10:10:43.847347+0000 (oldest deadline 2023-12-12T10:11:09.647657+0000) 2023-12-12T10:11:11.073 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:10 smithi173 systemd[1]: Stopping Ceph osd.11 for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:11.415 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.081+0000 7fc1bccf0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:11.415 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.081+0000 7fc1bccf0700 -1 osd.11 104 *** Got signal Terminated *** 2023-12-12T10:11:11.415 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.081+0000 7fc1bccf0700 -1 osd.11 104 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-12T10:11:11.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.413+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:11.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.413+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:11.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.413+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:11.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.413+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:11.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.413+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:11.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.413+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:11.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:11 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:11.413+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2023-12-12T10:10:43.847256+0000 front 2023-12-12T10:10:43.847347+0000 (oldest deadline 2023-12-12T10:11:09.647657+0000) 2023-12-12T10:11:12.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:12 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:12.412+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:12.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:12 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:12.412+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:12.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:12 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:12.412+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:12.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:12 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:12.412+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:12.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:12 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:12.412+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:12.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:12 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:12.412+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:12.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:12 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:12.412+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2023-12-12T10:10:43.847256+0000 front 2023-12-12T10:10:43.847347+0000 (oldest deadline 2023-12-12T10:11:09.647657+0000) 2023-12-12T10:11:13.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:13 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:13.455+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:13.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:13 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:13.455+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:13.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:13 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:13.455+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:13.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:13 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:13.455+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:13.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:13 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:13.455+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:13.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:13 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:13.455+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:13.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:13 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:13.455+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2023-12-12T10:10:43.847256+0000 front 2023-12-12T10:10:43.847347+0000 (oldest deadline 2023-12-12T10:11:09.647657+0000) 2023-12-12T10:11:14.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:14.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:14.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:14.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:14.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:14.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:14.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:14.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:14.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:14.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:14.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:14.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:14.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:14 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:14.461+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2023-12-12T10:10:43.847256+0000 front 2023-12-12T10:10:43.847347+0000 (oldest deadline 2023-12-12T10:11:09.647657+0000) 2023-12-12T10:11:15.823 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:15.476+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2023-12-12T10:09:47.036755+0000 front 2023-12-12T10:09:47.036709+0000 (oldest deadline 2023-12-12T10:10:10.536791+0000) 2023-12-12T10:11:15.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:15.476+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6804 osd.1 since back 2023-12-12T10:09:57.439186+0000 front 2023-12-12T10:09:57.439100+0000 (oldest deadline 2023-12-12T10:10:20.239671+0000) 2023-12-12T10:11:15.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:15.476+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6804 osd.2 since back 2023-12-12T10:10:06.140448+0000 front 2023-12-12T10:10:06.140487+0000 (oldest deadline 2023-12-12T10:10:30.840652+0000) 2023-12-12T10:11:15.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:15.476+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6812 osd.4 since back 2023-12-12T10:10:22.443683+0000 front 2023-12-12T10:10:22.443514+0000 (oldest deadline 2023-12-12T10:10:47.143551+0000) 2023-12-12T10:11:15.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:15.476+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.173:6812 osd.5 since back 2023-12-12T10:10:27.144057+0000 front 2023-12-12T10:10:27.143995+0000 (oldest deadline 2023-12-12T10:10:53.044087+0000) 2023-12-12T10:11:15.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:15.476+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2023-12-12T10:10:37.446390+0000 front 2023-12-12T10:10:37.446351+0000 (oldest deadline 2023-12-12T10:11:02.146359+0000) 2023-12-12T10:11:15.824 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:15 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11[133291]: 2023-12-12T10:11:15.476+0000 7fc1b52e3700 -1 osd.11 104 heartbeat_check: no reply from 172.21.15.111:6820 osd.7 since back 2023-12-12T10:10:43.847256+0000 front 2023-12-12T10:10:43.847347+0000 (oldest deadline 2023-12-12T10:11:09.647657+0000) 2023-12-12T10:11:16.866 INFO:journalctl@ceph.osd.11.smithi173.stdout:Dec 12 10:11:16 smithi173 bash[147483]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-osd-11 2023-12-12T10:11:18.128 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@osd.11.service' 2023-12-12T10:11:18.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:18.208 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-12-12T10:11:18.208 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-12-12T10:11:18.208 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.a 2023-12-12T10:11:18.566 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 10:11:18 smithi042 systemd[1]: Stopping Ceph mds.a for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:18.904 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 10:11:18 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-a[151506]: 2023-12-12T10:11:18.562+0000 7fa675cb5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:18.904 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 10:11:18 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-a[151506]: 2023-12-12T10:11:18.562+0000 7fa675cb5700 -1 mds.a *** got signal Terminated *** 2023-12-12T10:11:21.904 INFO:journalctl@ceph.mds.a.smithi042.stdout:Dec 12 10:11:21 smithi042 bash[277962]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-a 2023-12-12T10:11:21.988 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.a.service' 2023-12-12T10:11:22.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:22.026 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-12-12T10:11:22.026 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-12-12T10:11:22.026 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.d 2023-12-12T10:11:22.394 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 10:11:22 smithi042 systemd[1]: Stopping Ceph mds.d for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:22.654 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 10:11:22 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-d[150514]: 2023-12-12T10:11:22.391+0000 7f8743515700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:22.654 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 10:11:22 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-d[150514]: 2023-12-12T10:11:22.391+0000 7f8743515700 -1 mds.d *** got signal Terminated *** 2023-12-12T10:11:26.874 INFO:journalctl@ceph.mds.d.smithi042.stdout:Dec 12 10:11:26 smithi042 bash[278105]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-d 2023-12-12T10:11:26.880 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.d.service' 2023-12-12T10:11:26.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:26.915 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-12-12T10:11:26.915 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-12-12T10:11:26.916 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.g 2023-12-12T10:11:27.154 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 10:11:26 smithi042 systemd[1]: Stopping Ceph mds.g for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:27.654 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 10:11:27 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-g[151122]: 2023-12-12T10:11:27.286+0000 7f9dece27700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:27.654 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 10:11:27 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-g[151122]: 2023-12-12T10:11:27.286+0000 7f9dece27700 -1 mds.g *** got signal Terminated *** 2023-12-12T10:11:29.654 INFO:journalctl@ceph.mds.g.smithi042.stdout:Dec 12 10:11:29 smithi042 bash[278251]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-g 2023-12-12T10:11:29.850 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.g.service' 2023-12-12T10:11:29.886 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:29.886 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-12-12T10:11:29.886 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-12-12T10:11:29.887 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.j 2023-12-12T10:11:30.154 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 10:11:29 smithi042 systemd[1]: Stopping Ceph mds.j for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:30.654 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 10:11:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-j[150151]: 2023-12-12T10:11:30.252+0000 7f40814a3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:30.654 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 10:11:30 smithi042 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-j[150151]: 2023-12-12T10:11:30.252+0000 7f40814a3700 -1 mds.j *** got signal Terminated *** 2023-12-12T10:11:32.638 INFO:journalctl@ceph.mds.j.smithi042.stdout:Dec 12 10:11:32 smithi042 bash[278392]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-j 2023-12-12T10:11:32.764 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.j.service' 2023-12-12T10:11:32.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:32.800 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-12-12T10:11:32.800 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-12-12T10:11:32.800 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.b 2023-12-12T10:11:33.152 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 10:11:32 smithi111 systemd[1]: Stopping Ceph mds.b for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:33.498 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 10:11:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: 2023-12-12T10:11:33.148+0000 7f3700f40700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:33.498 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 10:11:33 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b[139649]: 2023-12-12T10:11:33.148+0000 7f3700f40700 -1 mds.b *** got signal Terminated *** 2023-12-12T10:11:37.694 INFO:journalctl@ceph.mds.b.smithi111.stdout:Dec 12 10:11:37 smithi111 bash[147126]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-b 2023-12-12T10:11:37.699 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.b.service' 2023-12-12T10:11:37.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:37.737 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-12-12T10:11:37.738 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-12-12T10:11:37.738 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.e 2023-12-12T10:11:37.998 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 10:11:37 smithi111 systemd[1]: Stopping Ceph mds.e for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:38.498 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 10:11:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-e[138965]: 2023-12-12T10:11:38.105+0000 7f6877896700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:38.498 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 10:11:38 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-e[138965]: 2023-12-12T10:11:38.105+0000 7f6877896700 -1 mds.e *** got signal Terminated *** 2023-12-12T10:11:41.748 INFO:journalctl@ceph.mds.e.smithi111.stdout:Dec 12 10:11:41 smithi111 bash[147278]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-e 2023-12-12T10:11:41.775 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.e.service' 2023-12-12T10:11:41.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:41.809 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-12-12T10:11:41.810 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-12-12T10:11:41.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.h 2023-12-12T10:11:42.182 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 10:11:41 smithi111 systemd[1]: Stopping Ceph mds.h for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:42.498 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 10:11:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-h[139308]: 2023-12-12T10:11:42.179+0000 7fde3bebc700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:42.498 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 10:11:42 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-h[139308]: 2023-12-12T10:11:42.179+0000 7fde3bebc700 -1 mds.h *** got signal Terminated *** 2023-12-12T10:11:45.653 INFO:journalctl@ceph.mds.h.smithi111.stdout:Dec 12 10:11:45 smithi111 bash[147423]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-h 2023-12-12T10:11:45.842 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.h.service' 2023-12-12T10:11:45.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:45.877 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-12-12T10:11:45.877 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-12-12T10:11:45.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.k 2023-12-12T10:11:46.247 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 10:11:45 smithi111 systemd[1]: Stopping Ceph mds.k for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:46.748 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 10:11:46 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-k[138626]: 2023-12-12T10:11:46.245+0000 7f9ef020b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:46.748 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 10:11:46 smithi111 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-k[138626]: 2023-12-12T10:11:46.245+0000 7f9ef020b700 -1 mds.k *** got signal Terminated *** 2023-12-12T10:11:49.707 INFO:journalctl@ceph.mds.k.smithi111.stdout:Dec 12 10:11:49 smithi111 bash[147563]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-k 2023-12-12T10:11:49.713 DEBUG:teuthology.orchestra.run.smithi111:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.k.service' 2023-12-12T10:11:49.747 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:49.748 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-12-12T10:11:49.748 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-12-12T10:11:49.748 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.c 2023-12-12T10:11:50.073 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 10:11:49 smithi173 systemd[1]: Stopping Ceph mds.c for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:50.573 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 10:11:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-c[140159]: 2023-12-12T10:11:50.106+0000 7f43dde60700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:50.573 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 10:11:50 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-c[140159]: 2023-12-12T10:11:50.106+0000 7f43dde60700 -1 mds.c *** got signal Terminated *** 2023-12-12T10:11:53.674 INFO:journalctl@ceph.mds.c.smithi173.stdout:Dec 12 10:11:53 smithi173 bash[147747]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-c 2023-12-12T10:11:53.867 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.c.service' 2023-12-12T10:11:53.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:53.903 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-12-12T10:11:53.903 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-12-12T10:11:53.903 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.f 2023-12-12T10:11:54.277 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 10:11:53 smithi173 systemd[1]: Stopping Ceph mds.f for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:54.573 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 10:11:54 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-f[139821]: 2023-12-12T10:11:54.274+0000 7f59e7b42700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:54.573 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 10:11:54 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-f[139821]: 2023-12-12T10:11:54.274+0000 7f59e7b42700 -1 mds.f *** got signal Terminated *** 2023-12-12T10:11:57.649 INFO:journalctl@ceph.mds.f.smithi173.stdout:Dec 12 10:11:57 smithi173 bash[147885]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-f 2023-12-12T10:11:57.835 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.f.service' 2023-12-12T10:11:57.870 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:11:57.870 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-12-12T10:11:57.870 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-12-12T10:11:57.870 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.i 2023-12-12T10:11:58.251 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:11:57 smithi173 systemd[1]: Stopping Ceph mds.i for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:11:58.573 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:11:58 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-i[140500]: 2023-12-12T10:11:58.248+0000 7f6ba519c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:11:58.573 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:11:58 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-i[140500]: 2023-12-12T10:11:58.248+0000 7f6ba519c700 -1 mds.i *** got signal Terminated *** 2023-12-12T10:12:01.616 INFO:journalctl@ceph.mds.i.smithi173.stdout:Dec 12 10:12:01 smithi173 bash[148024]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-i 2023-12-12T10:12:01.799 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.i.service' 2023-12-12T10:12:01.832 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:12:01.833 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-12-12T10:12:01.833 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-12-12T10:12:01.833 DEBUG:teuthology.orchestra.run.smithi173:> sudo systemctl stop ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.l 2023-12-12T10:12:02.073 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 10:12:01 smithi173 systemd[1]: Stopping Ceph mds.l for 968ccaec-98d3-11ee-95a3-87774f69a715... 2023-12-12T10:12:02.573 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 10:12:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-l[139473]: 2023-12-12T10:12:02.207+0000 7f563cbee700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-12T10:12:02.573 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 10:12:02 smithi173 ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-l[139473]: 2023-12-12T10:12:02.207+0000 7f563cbee700 -1 mds.l *** got signal Terminated *** 2023-12-12T10:12:05.823 INFO:journalctl@ceph.mds.l.smithi173.stdout:Dec 12 10:12:05 smithi173 bash[148161]: ceph-968ccaec-98d3-11ee-95a3-87774f69a715-mds-l 2023-12-12T10:12:05.869 DEBUG:teuthology.orchestra.run.smithi173:> sudo pkill -f 'journalctl -f -n 0 -u ceph-968ccaec-98d3-11ee-95a3-87774f69a715@mds.l.service' 2023-12-12T10:12:05.903 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-12T10:12:05.903 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-12-12T10:12:05.903 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 --force --keep-logs 2023-12-12T10:12:06.260 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T10:12:08.537 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 --force --keep-logs 2023-12-12T10:12:08.878 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T10:12:10.452 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 --force --keep-logs 2023-12-12T10:12:10.799 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T10:12:12.374 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-12T10:12:12.412 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-12T10:12:12.448 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-12T10:12:12.477 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-12T10:12:12.479 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/crash to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/remote/smithi042/crash 2023-12-12T10:12:12.480 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/crash -- . 2023-12-12T10:12:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/crash: Cannot open: No such file or directory 2023-12-12T10:12:12.525 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now 2023-12-12T10:12:12.527 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/crash to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/remote/smithi111/crash 2023-12-12T10:12:12.528 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/crash -- . 2023-12-12T10:12:12.571 INFO:teuthology.orchestra.run.smithi111.stderr:tar: /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/crash: Cannot open: No such file or directory 2023-12-12T10:12:12.572 INFO:teuthology.orchestra.run.smithi111.stderr:tar: Error is not recoverable: exiting now 2023-12-12T10:12:12.573 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/crash to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/remote/smithi173/crash 2023-12-12T10:12:12.574 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/crash -- . 2023-12-12T10:12:12.615 INFO:teuthology.orchestra.run.smithi173.stderr:tar: /var/lib/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/crash: Cannot open: No such file or directory 2023-12-12T10:12:12.615 INFO:teuthology.orchestra.run.smithi173.stderr:tar: Error is not recoverable: exiting now 2023-12-12T10:12:12.616 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-12T10:12:12.617 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-12T10:12:12.663 INFO:teuthology.orchestra.run.smithi042.stderr:grep: /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph.log: No such file or directory 2023-12-12T10:12:12.666 INFO:tasks.cephadm:Compressing logs... 2023-12-12T10:12:12.666 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-12T10:12:12.668 DEBUG:teuthology.orchestra.run.smithi111:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-12T10:12:12.671 DEBUG:teuthology.orchestra.run.smithi173:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-12T10:12:12.699 INFO:teuthology.orchestra.run.smithi111.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-12T10:12:12.699 INFO:teuthology.orchestra.run.smithi111.stderr:: No such file or directory 2023-12-12T10:12:12.700 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.10.log 2023-12-12T10:12:12.701 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.h.log 2023-12-12T10:12:12.701 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.e.log 2023-12-12T10:12:12.701 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.10.log: /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.4.log 2023-12-12T10:12:12.701 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.1.log 2023-12-12T10:12:12.701 INFO:teuthology.orchestra.run.smithi111.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-12T10:12:12.702 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.k.log 2023-12-12T10:12:12.706 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-12T10:12:12.707 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-12T10:12:12.712 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2023-12-12T10:12:12.712 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.0.log 2023-12-12T10:12:12.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.g.log 2023-12-12T10:12:12.713 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.a.log 2023-12-12T10:12:12.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.0.log: /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.6.log 2023-12-12T10:12:12.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.3.log 2023-12-12T10:12:12.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.d.log 2023-12-12T10:12:12.713 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mgr.x.log 2023-12-12T10:12:12.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.9.log 2023-12-12T10:12:12.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-volume.log 2023-12-12T10:12:12.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.j.log 2023-12-12T10:12:12.714 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mon.a.log 2023-12-12T10:12:12.715 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-volume.log 2023-12-12T10:12:12.718 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.j.log: /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mon.a.log: 93.9% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2023-12-12T10:12:12.720 INFO:teuthology.orchestra.run.smithi173.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-12T10:12:12.720 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-12-12T10:12:12.721 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.k.log: /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.7.log 2023-12-12T10:12:12.721 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.7.log: 91.2% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-volume.log.gz 2023-12-12T10:12:12.722 INFO:teuthology.orchestra.run.smithi111.stderr:gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mon.b.log 2023-12-12T10:12:12.730 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.5.log 2023-12-12T10:12:12.730 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.2.log 2023-12-12T10:12:12.730 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.11.log 2023-12-12T10:12:12.730 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mgr.z.log 2023-12-12T10:12:12.730 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.8.log 2023-12-12T10:12:12.730 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.f.log 2023-12-12T10:12:12.731 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.c.log 2023-12-12T10:12:12.731 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.i.log 2023-12-12T10:12:12.731 INFO:teuthology.orchestra.run.smithi173.stderr:gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mon.c.log 2023-12-12T10:12:12.731 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.c.log: /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-volume.log 2023-12-12T10:12:12.731 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.l.log 2023-12-12T10:12:12.732 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.b.log 2023-12-12T10:12:12.734 INFO:teuthology.orchestra.run.smithi173.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-volume.log: /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.l.log: 91.7% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mgr.z.log.gz 2023-12-12T10:12:12.751 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mgr.y.log 2023-12-12T10:12:12.779 INFO:teuthology.orchestra.run.smithi111.stderr:/var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mgr.y.log: 91.7% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mgr.y.log.gz 2023-12-12T10:12:13.056 INFO:teuthology.orchestra.run.smithi173.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-12T10:12:13.070 INFO:teuthology.orchestra.run.smithi042.stderr: 91.2% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-volume.log.gz 2023-12-12T10:12:13.182 INFO:teuthology.orchestra.run.smithi111.stderr: 92.0% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mon.b.log.gz 2023-12-12T10:12:13.241 INFO:teuthology.orchestra.run.smithi042.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-12-12T10:12:13.526 INFO:teuthology.orchestra.run.smithi042.stderr: 94.5% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.d.log.gz 2023-12-12T10:12:13.754 INFO:teuthology.orchestra.run.smithi042.stderr: 94.5% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.j.log.gz 2023-12-12T10:12:13.822 INFO:teuthology.orchestra.run.smithi042.stderr: 89.8% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mgr.x.log.gz 2023-12-12T10:12:13.862 INFO:teuthology.orchestra.run.smithi173.stderr: 92.1% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mon.c.log.gz 2023-12-12T10:12:14.186 INFO:teuthology.orchestra.run.smithi173.stderr: 91.2% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-volume.log.gz 2023-12-12T10:12:14.782 INFO:teuthology.orchestra.run.smithi042.stderr: 90.5% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mon.a.log.gz 2023-12-12T10:12:20.771 INFO:teuthology.orchestra.run.smithi111.stderr: 94.9% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.e.log.gz 2023-12-12T10:12:22.092 INFO:teuthology.orchestra.run.smithi042.stderr: 93.6% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.g.log.gz 2023-12-12T10:12:23.087 INFO:teuthology.orchestra.run.smithi111.stderr: 93.8% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.k.log.gz 2023-12-12T10:12:25.212 INFO:teuthology.orchestra.run.smithi042.stderr: 93.8% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.a.log.gz 2023-12-12T10:12:32.877 INFO:teuthology.orchestra.run.smithi111.stderr: 91.8% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.10.log.gz 2023-12-12T10:12:32.946 INFO:teuthology.orchestra.run.smithi173.stderr: 94.3% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.l.log.gz 2023-12-12T10:12:37.539 INFO:teuthology.orchestra.run.smithi042.stderr: 91.5% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.0.log.gz 2023-12-12T10:12:40.990 INFO:teuthology.orchestra.run.smithi042.stderr: 91.5% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.3.log.gz 2023-12-12T10:12:41.194 INFO:teuthology.orchestra.run.smithi111.stderr: 91.7% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.1.log.gz 2023-12-12T10:12:43.496 INFO:teuthology.orchestra.run.smithi042.stderr: 91.5% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.6.log.gz 2023-12-12T10:12:43.709 INFO:teuthology.orchestra.run.smithi173.stderr: 91.7% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.8.log.gz 2023-12-12T10:12:44.519 INFO:teuthology.orchestra.run.smithi042.stderr: 91.6% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.9.log.gz 2023-12-12T10:12:44.521 INFO:teuthology.orchestra.run.smithi042.stderr: 2023-12-12T10:12:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:real 0m31.837s 2023-12-12T10:12:44.521 INFO:teuthology.orchestra.run.smithi042.stderr:user 2m8.924s 2023-12-12T10:12:44.522 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m3.902s 2023-12-12T10:12:45.959 INFO:teuthology.orchestra.run.smithi111.stderr: 91.6% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.4.log.gz 2023-12-12T10:12:47.411 INFO:teuthology.orchestra.run.smithi111.stderr: 91.5% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.7.log.gz 2023-12-12T10:12:52.269 INFO:teuthology.orchestra.run.smithi173.stderr: 91.6% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.11.log.gz 2023-12-12T10:12:57.322 INFO:teuthology.orchestra.run.smithi173.stderr: 91.6% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.2.log.gz 2023-12-12T10:13:01.348 INFO:teuthology.orchestra.run.smithi173.stderr: 93.3% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.f.log.gz 2023-12-12T10:13:04.072 INFO:teuthology.orchestra.run.smithi111.stderr: 93.5% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.h.log.gz 2023-12-12T10:13:06.679 INFO:teuthology.orchestra.run.smithi173.stderr: 91.4% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-osd.5.log.gz 2023-12-12T10:13:13.445 INFO:teuthology.orchestra.run.smithi173.stderr: 93.3% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.c.log.gz 2023-12-12T10:13:20.978 INFO:teuthology.orchestra.run.smithi111.stderr: 93.0% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.b.log.gz 2023-12-12T10:13:20.981 INFO:teuthology.orchestra.run.smithi111.stderr: 2023-12-12T10:13:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:real 1m8.293s 2023-12-12T10:13:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:user 4m6.473s 2023-12-12T10:13:20.981 INFO:teuthology.orchestra.run.smithi111.stderr:sys 0m7.435s 2023-12-12T10:13:26.491 INFO:teuthology.orchestra.run.smithi173.stderr: 93.6% -- replaced with /var/log/ceph/968ccaec-98d3-11ee-95a3-87774f69a715/ceph-mds.i.log.gz 2023-12-12T10:13:26.494 INFO:teuthology.orchestra.run.smithi173.stderr: 2023-12-12T10:13:26.494 INFO:teuthology.orchestra.run.smithi173.stderr:real 1m13.806s 2023-12-12T10:13:26.494 INFO:teuthology.orchestra.run.smithi173.stderr:user 4m45.123s 2023-12-12T10:13:26.494 INFO:teuthology.orchestra.run.smithi173.stderr:sys 0m9.181s 2023-12-12T10:13:26.495 INFO:tasks.cephadm:Archiving logs... 2023-12-12T10:13:26.496 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/remote/smithi042/log 2023-12-12T10:13:26.497 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-12T10:13:45.953 DEBUG:teuthology.misc:Transferring archived files from smithi111:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/remote/smithi111/log 2023-12-12T10:13:45.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-12T10:14:18.037 DEBUG:teuthology.misc:Transferring archived files from smithi173:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/remote/smithi173/log 2023-12-12T10:14:18.038 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-12T10:14:54.536 INFO:tasks.cephadm:Removing cluster... 2023-12-12T10:14:54.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 --force 2023-12-12T10:14:54.910 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T10:14:55.598 DEBUG:teuthology.orchestra.run.smithi111:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 --force 2023-12-12T10:14:55.959 INFO:teuthology.orchestra.run.smithi111.stdout:Deleting cluster with fsid: 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T10:14:56.717 DEBUG:teuthology.orchestra.run.smithi173:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 968ccaec-98d3-11ee-95a3-87774f69a715 --force 2023-12-12T10:14:57.061 INFO:teuthology.orchestra.run.smithi173.stdout:Deleting cluster with fsid: 968ccaec-98d3-11ee-95a3-87774f69a715 2023-12-12T10:14:57.841 INFO:tasks.cephadm:Removing cephadm ... 2023-12-12T10:14:57.841 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-12T10:14:57.864 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-12T10:14:57.888 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-12T10:14:57.909 INFO:tasks.cephadm:Teardown complete 2023-12-12T10:14:57.909 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-12T10:14:58.455 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-12T10:14:58.455 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-12T10:14:58.458 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-12T10:14:58.461 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-12T10:14:58.572 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-12T10:14:58.696 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-12T10:14:58.696 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T10:14:58.699 DEBUG:teuthology.orchestra.run.smithi111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T10:14:58.703 DEBUG:teuthology.orchestra.run.smithi173:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-12T10:14:58.717 INFO:teuthology.orchestra.run.smithi042.stderr:bash: ntpq: command not found 2023-12-12T10:14:58.719 INFO:teuthology.orchestra.run.smithi111.stderr:bash: ntpq: command not found 2023-12-12T10:14:58.720 INFO:teuthology.orchestra.run.smithi173.stderr:bash: ntpq: command not found 2023-12-12T10:14:58.725 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T10:14:58.726 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2023-12-12T10:14:58.726 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +1920us[+1923us] +/- 69ms 2023-12-12T10:14:58.726 INFO:teuthology.orchestra.run.smithi111.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -967us[ -965us] +/- 56ms 2023-12-12T10:14:58.726 INFO:teuthology.orchestra.run.smithi111.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -683us[ -680us] +/- 82ms 2023-12-12T10:14:58.726 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T10:14:58.729 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T10:14:58.729 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2023-12-12T10:14:58.729 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 139 +1977us[+1952us] +/- 99ms 2023-12-12T10:14:58.729 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 -931us[ -936us] +/- 56ms 2023-12-12T10:14:58.729 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -646us[ -651us] +/- 82ms 2023-12-12T10:14:58.729 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T10:14:58.745 INFO:teuthology.orchestra.run.smithi173.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-12T10:14:58.746 INFO:teuthology.orchestra.run.smithi173.stdout:=============================================================================== 2023-12-12T10:14:58.746 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 9 +1922us[+1923us] +/- 69ms 2023-12-12T10:14:58.746 INFO:teuthology.orchestra.run.smithi173.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -966us[ -964us] +/- 56ms 2023-12-12T10:14:58.746 INFO:teuthology.orchestra.run.smithi173.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 9 -691us[ -690us] +/- 82ms 2023-12-12T10:14:58.746 INFO:teuthology.orchestra.run.smithi173.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-12T10:14:58.747 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-12T10:14:58.883 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-12T10:14:58.884 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-12T10:14:58.924 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-12T10:14:59.167 DEBUG:teuthology.orchestra.run.smithi111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-12T10:14:59.270 DEBUG:teuthology.orchestra.run.smithi173:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-12T10:14:59.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T10:14:59.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T10:14:59.434 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T10:14:59.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T10:14:59.468 DEBUG:teuthology.orchestra.run.smithi173:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-12T10:14:59.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T10:14:59.501 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-12T10:14:59.513 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-12T10:14:59.524 INFO:teuthology.task.internal:Duration was 2371.505717 seconds 2023-12-12T10:14:59.525 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-12T10:14:59.538 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-12T10:14:59.539 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-12T10:14:59.541 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-12T10:14:59.543 DEBUG:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-12T10:14:59.590 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T10:14:59.617 INFO:teuthology.orchestra.run.smithi173.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T10:14:59.653 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-12T10:14:59.953 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-12T10:14:59.953 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2023-12-12T10:14:59.953 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-12T10:14:59.993 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi111.front.sepia.ceph.com 2023-12-12T10:14:59.994 DEBUG:teuthology.orchestra.run.smithi111:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-12T10:15:00.017 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi173.front.sepia.ceph.com 2023-12-12T10:15:00.017 DEBUG:teuthology.orchestra.run.smithi173:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-12T10:15:00.067 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-12T10:15:00.068 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-12T10:15:00.070 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-12T10:15:00.072 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-12T10:15:00.146 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-12T10:15:00.147 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-12T10:15:00.150 DEBUG:teuthology.orchestra.run.smithi111:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-12T10:15:00.153 DEBUG:teuthology.orchestra.run.smithi173:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-12T10:15:00.276 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-12T10:15:00.288 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-12T10:15:00.289 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-12T10:15:00.326 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-12T10:15:00.419 DEBUG:teuthology.orchestra.run.smithi173:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-12T10:15:00.446 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-12T10:15:00.459 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-12T10:15:00.461 DEBUG:teuthology.orchestra.run.smithi111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-12T10:15:00.464 DEBUG:teuthology.orchestra.run.smithi173:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-12T10:15:00.488 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = core 2023-12-12T10:15:00.494 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2023-12-12T10:15:00.512 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = core 2023-12-12T10:15:00.580 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-12T10:15:00.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T10:15:00.597 DEBUG:teuthology.orchestra.run.smithi111:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-12T10:15:00.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T10:15:00.613 DEBUG:teuthology.orchestra.run.smithi173:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-12T10:15:00.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-12T10:15:00.637 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-12T10:15:00.650 INFO:teuthology.task.internal:Transferring archived files... 2023-12-12T10:15:00.651 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/remote/smithi042 2023-12-12T10:15:00.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-12T10:15:00.697 DEBUG:teuthology.misc:Transferring archived files from smithi111:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/remote/smithi111 2023-12-12T10:15:00.697 DEBUG:teuthology.orchestra.run.smithi111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-12T10:15:00.738 DEBUG:teuthology.misc:Transferring archived files from smithi173:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220/remote/smithi173 2023-12-12T10:15:00.738 DEBUG:teuthology.orchestra.run.smithi173:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-12T10:15:00.779 INFO:teuthology.task.internal:Removing archive directory... 2023-12-12T10:15:00.780 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-12T10:15:00.782 DEBUG:teuthology.orchestra.run.smithi111:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-12T10:15:00.785 DEBUG:teuthology.orchestra.run.smithi173:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-12T10:15:00.830 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-12T10:15:00.844 INFO:teuthology.task.internal:Not uploading archives. 2023-12-12T10:15:00.845 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-12T10:15:00.860 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-12T10:15:00.860 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-12T10:15:00.862 DEBUG:teuthology.orchestra.run.smithi111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-12T10:15:00.865 DEBUG:teuthology.orchestra.run.smithi173:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-12T10:15:00.877 INFO:teuthology.orchestra.run.smithi042.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 12 10:15 /home/ubuntu/cephtest 2023-12-12T10:15:00.881 INFO:teuthology.orchestra.run.smithi111.stdout: 262181 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 12 10:15 /home/ubuntu/cephtest 2023-12-12T10:15:00.887 INFO:teuthology.orchestra.run.smithi173.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 12 10:15 /home/ubuntu/cephtest 2023-12-12T10:15:00.888 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-12T10:15:00.901 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-12T10:15:00.926 INFO:teuthology.nuke:Checking targets against current locks 2023-12-12T10:15:00.956 DEBUG:teuthology.nuke:shortname: smithi042 2023-12-12T10:15:00.957 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-12T10:15:00.981 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:29:39.006552', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-12T10:15:00.985 DEBUG:teuthology.nuke:shortname: smithi111 2023-12-12T10:15:00.985 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-12T10:15:01.015 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:29:39.005028', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-12T10:15:01.016 DEBUG:teuthology.nuke:shortname: smithi173 2023-12-12T10:15:01.016 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-12T10:15:01.039 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi173.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-12-12_08:46:52-fs-wip-mchangir-testing-20231205.064351-testing-default-smithi/7488220', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-12 09:29:39.007693', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-12-12T10:15:01.049 INFO:teuthology.orchestra.console.smithi042:Power off 2023-12-12T10:15:01.050 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-12T10:15:01.110 INFO:teuthology.orchestra.console.smithi111:Power off 2023-12-12T10:15:01.110 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-12T10:15:01.149 INFO:teuthology.orchestra.console.smithi173:Power off 2023-12-12T10:15:01.150 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-12T10:15:01.168 DEBUG:teuthology.orchestra.console.smithi042:power off output: Chassis Power Control: Down/Off 2023-12-12T10:15:01.169 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:01.182 DEBUG:teuthology.orchestra.console.smithi173:power off output: Chassis Power Control: Down/Off 2023-12-12T10:15:01.182 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:01.194 DEBUG:teuthology.orchestra.console.smithi111:power off output: Chassis Power Control: Down/Off 2023-12-12T10:15:01.194 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:01.205 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2023-12-12T10:15:01.209 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2023-12-12T10:15:01.218 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2023-12-12T10:15:05.207 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:05.216 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:05.224 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:05.318 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is on 2023-12-12T10:15:05.326 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is on 2023-12-12T10:15:05.335 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2023-12-12T10:15:09.322 DEBUG:teuthology.orchestra.console.smithi042:pexpect command: ipmitool -H smithi042.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:09.331 DEBUG:teuthology.orchestra.console.smithi173:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:09.341 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:09.434 DEBUG:teuthology.orchestra.console.smithi042:check power output: Chassis Power is off 2023-12-12T10:15:09.443 DEBUG:teuthology.orchestra.console.smithi173:check power output: Chassis Power is off 2023-12-12T10:15:09.452 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is on 2023-12-12T10:15:09.536 INFO:teuthology.orchestra.console.smithi042:Power off completed 2023-12-12T10:15:09.544 INFO:teuthology.orchestra.console.smithi173:Power off completed 2023-12-12T10:15:13.454 DEBUG:teuthology.orchestra.console.smithi111:pexpect command: ipmitool -H smithi111.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-12T10:15:13.566 DEBUG:teuthology.orchestra.console.smithi111:check power output: Chassis Power is off 2023-12-12T10:15:13.667 INFO:teuthology.orchestra.console.smithi111:Power off completed 2023-12-12T10:15:13.774 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} duration: 2371.505717277527 failure_reason: 'Command failed (workunit test fs/misc/general_vxattrs.sh) on smithi042 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f1fbf3c583d03e19f5e58eb618e78d8e10d075f9 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh''' flavor: default owner: mchangir sentry_event: https://sentry.ceph.com/organizations/ceph/?query=b02660a13c2240ecbf2796147cb87b96 status: fail success: false 2023-12-12T10:15:13.774 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-12T10:15:13.864 INFO:teuthology.run:FAIL